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Abstract 

This  paper  proposes  an  approach  for  measuring  the  return  on  Information  Technology  (IT) 
investments.  A  review  of  existing  methods  suggests  the  difficulty  in  adequately  measuring 
the  returns  of  IT  at  various  levels  of  analysis  (e.g.,  firm  or  process  level).  To  address  this 
issue,  this  study  aims  to  develop  a  method  for  allocating  the  revenue  and  cost  of  IT  initiatives 
at  any  level  of  analysis  using  a  common  unit  of  measurement. 

Following  the  knowledge-based  view  (KBV),  this  paper  proposes  an  analytic  method  for 
measuring  the  historical  revenue  and  cost  of  IT  investments  by  estimating  the  amount  of 
knowledge  necessary  to  generate  a  common  unit  of  output  from  any  business  process.  The 
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amount  of  required  knowledge  is  operationalized  using  the  ‘average  learning  time’  measure. 
The  proposed  operationalization  is  illustrated  with  a  practical  case  example. 

The  proposed  KBV  approach  is  extended  specifically  for  IT  resources,  allowing  us  to  assess 
the  Return  on  IT  (ROIT)  using  a  typical  productivity  ratio  (similar  to  ROI  or  ROA)  that 
accurately  captures  the  true  business  value  of  IT  (despite  any  complementarities)  at  virtually 
any  level  of  analysis. 
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Theory,  Knowledge-Based  View,  IT  Investments,  Thermodynamics,  Learning  Time 


Introduction 

There  have  been  numerous  approaches  to  assessing  the  performance  impact  of  IT 
investments,  both  at  the  firm  and  the  process  levels  (see  Kohli  and  Devaraj,  2003  for  a 
review).  At  the  firm  level,  the  key  question  is  how  IT  initiatives  affect  firm  performance 
(Brynjolfsson  and  Hitt,  1993;  Brynjolfsson  and  Yang,  1996;  Dos  Santos,  Peffers  and  Mauer, 
1993;  Im,  Dow  and  Grover,  2001;  Richardson  and  Zmud,  2002;  Subramani  and  Walden, 
2001).  At  the  process  level,  the  question  is  about  the  impact  of  IT  investments  on  specific 
processes  (e.g.,  Barua,  Kriebel  and  Mukhopadhyay,  1995;  El  Sawy  2001;  Ray,  Muhanna  and 
Barney,  2005).  In  general,  the  literature  has  shown  that  IT  may  often  facilitate  firm 
performance,  yet  not  conclusively.  Besides  measurement  concerns,  lag  effects,  poor  IT 
investments,  and  inappropriate  IT  implementation  and  use,  the  infamous  “IT  productivity 
paradox”  may  be  due  to  the  fact  that  the  output  of  IT  initiatives  is  perhaps  better  observed  at 
the  process  level  (versus  the  firm  level)  (Barua  et  al.,  1995;  Kohli  and  Devaraj,  2003;  Ray  et 
al.,  2005).  In  other  words,  IT-driven  firm  impacts  may  not  readily  materialize  because  of 
various  firm-level,  industry-level,  or  other  contingencies  (Melville,  Kraemer  and  Gurbaxani, 
2004).  To  address  the  inconclusiveness  regarding  the  returns  on  IT  investments,  this  study 
proposes  that  a  common  measurement  unit  is  needed  to  describe  the  output  of  business 
processes  and  allocate  the  firm’s  revenue  and  cost,  regardless  of  the  chosen  level  of 
analysis  (process  or  firm  level). 

To  justify  enormous  IT  investments,  there  should  be  some  relationship  between  the  impact  of 
IT  investments  and  both  specific  firm  processes  and  overall  firm  performance.  However, 
many  other  variables  that  affect  firm  performance  (e.g.,  firm  size,  industry  structure,  new 
technologies,  and  government  regulations)  make  it  difficult  to  infer  that  a  specific  IT 
investment  has  resulted  in  a  specific  change  in  a  firm  process  or  overall  firm  performance 
(Bharadwaj,  2000).  Another  difficulty  is  that  IT  inputs  are  often  intertwined  with 
complementary  factors  (Barua  et  al.,  1995;  Brynjolfsson  and  Yang,  1996;  Brynjolfsson  and 
Hitt,  1998).  Provided  that  the  cost  and  the  revenue  due  to  IT  (plus  other  complementary 
factors)  could  be  specified  at  a  given  point  in  time,  it  would  be  possible  to  establish  an 
independent  return  ratio  for  productive  assets,  including  IT.  Given  that  this  approach  has  not 
yet  been  proposed  in  the  literature,  this  study  aims  to  introduce  an  analytic  method  for 
measuring  the  performance  impacts  of  IT  resources  by  ex-post  allocation  of  historical 
revenue  and  cost  to  IT  investments,  regardless  of  potential  complementarities  with 
intertwined  factors.1 


1  Given  that  IT  resources  are  often  intertwined  with  other  factors,  the  theory  of  complementarities 
(e.g.,  Barua  et  al.  1995)  suggests  that  it  is  difficult  to  perfectly  distinguish  between  IT  and 

200  Journal  of  the  Association  for  Information  Systems,  Vol.  6,  No.  1,  pp.  1 99-226/July  2005 


Pavlou  et  al. /Measuring  the  Return  on  IT 


The  proposed  approach  assumes  that  it  is  possible  to  describe  the  outputs  of  all  firm 
processes  in  common  units  by  employing  a  knowledge  metaphor.  The  basic  assumption  is 
that  processes  are  surrogates  for  their  respective  outputs,  and  that  by  using  a  common 
language  based  on  process  knowledge  estimates,  it  is  possible  to  describe  the  outputs  of 
processes  in  common  units.  Following  the  knowledge-based  view  (Kogut  and  Zander,  1996; 
Nahapiet  and  Ghoshal,  1998;  Nonaka,  1994;  Zander  and  Kogut,  1995),  the  assumption  is 
that  knowledge  is  required  to  produce  outputs  of  a  process,  and  it  is  possible  to  describe  the 
output  in  common  units.  The  proposed  conceptualization  is  based  on  the  relationship 
between  the  knowledge  required  to  produce  a  given  firm  output  (whether  that  knowledge  is 
contained  in  human  or  IT  resources),  and  the  time  it  would  take  the  “average”  learner  to 
acquire  that  knowledge  to  produce  the  output.  The  longer  it  takes  the  common  reference 
point  (the  average  learner)  to  learn  how  to  produce  a  given  output,  the  more  complex  is  the 
knowledge  required.  In  this  fashion,  the  operational  definition  of  knowledge  is  defined  in 
terms  of  learning  time.  Since  learning  time  is  verifiable  through  various  objective  means,  the 
resulting  estimates  can  be  checked  for  their  reliability.  The  proposed  operationalization  - 
average  learning  time  -  (corrected  for  biases  using  normal  sampling  and  ensuring  reliability 
of  estimates)  can  thus  be  measured  in  common  units  of  time.  Learning  time  is  proposed  as  a 
surrogate  for  common  units  of  output,  and  it  can  be  used  for  allocating  the  firm’s  revenue  and 
cost  to  the  common  units  of  output.2 

In  practice,  to  achieve  an  accurate  estimate  of  the  knowledge  embedded  in  IT  investments 
using  the  same  descriptive  language  of  learning  time,  it  is  also  possible  to  observe  the 
outputs  of  IT  and  estimate  how  long  it  would  take  the  average  learner  to  acquire  the  required 
knowledge  to  produce  the  desired  output.  For  example,  in  the  case  study  reviewed  in  this 
paper  (Section  4),  SBC  Telecom  documented  how  to  produce  the  outputs  of  the  various  IT 
investments  in  the  event  of  IT  failure.  This  provides  a  convenient  way  to  assess  how  long  the 
average  learner  would  take  to  learn  how  to  produce  the  outputs  of  IT. 

By  being  able  to  describe  process  outputs  in  common  measurement  units  based  on  their 
knowledge  complexity,  it  then  becomes  possible  to  first  allocate  the  firm’s  revenue  to  the 
common  units  of  output  produced  by  the  knowledge  embedded  in  productive  assets  (e.g., 
employees  or  IT  assets)  of  each  process.  In  other  words,  the  proposed  approach  allocates 
the  price  paid  for  these  common  units  of  output  (i.e.,  the  historical  revenue)  across  the  firm’s 
various  processes.  Second,  given  that  outputs  are  described  in  common  measurement  units, 
it  is  also  possible  to  get  estimates  for  the  cost  per  unit  of  output.  In  the  knowledge  metaphor, 
the  time  it  takes  to  master  the  knowledge  to  produce  a  common  unit  of  output  is  a  convenient 
way  to  estimate  cost.  More  specifically,  the  execution  time  multiplied  by  the  cost  of  the 
productive  assets  (i.e.,  employees  and  IT  assets)  provides  a  way  to  estimate  cost  using 
common  units  of  output. 


complementary  factors  (due  to  non-linearities).  To  address  this  issue,  this  method  allocates  revenue 
and  cost  to  the  joint  factor  (that  captures  all  complementarities),  and  it  then  distributes  the  cost  and 
revenue  among  its  constituents. 

2  It  is  assumed  that  when  customers  pay  a  certain  price  for  a  firm’s  output  (e.g.,  a  product),  they  pay 
for  the  all  the  outputs  of  the  firm  at  that  point  in  time.  For  example,  when  you  purchase  an  automobile, 
you  pay  a  price  that  is  distributed  across  all  processes  (e.g.,  product  development,  manufacturing, 
quality  control,  finance,  marketing,  sales,  accounting)  used  to  produce  the  automobile.  This  historical 
revenue  and  its  associated  cost  is  what  the  proposed  method  aims  to  allocate. 
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Applied  to  IT  investments,  we  propose  an  approach  for  estimating  the  Return  on  IT  (ROIT) 
investments  by  allocating  historical  revenue  and  cost  data  to  a  firm’s  outputs  described  in 
common  units.  Accounting  for  the  revenue  and  cost  of  IT  investments  (and  since  revenue 
and  cost  are  derived  independently),  it  thus  becomes  possible  to  calculate  a  productivity 
measure  of  the  firm’s  IT  (ROIT).  Generating  a  return-over-cost  ratio  specifically  for  IT 
investments  would  create  a  typical  productivity  ratio  (similar  to  ROI  or  ROA)  that  would 
enable  an  accurate  assessment  of  the  value  added  by  IT  investments.  It  would  also  make  it 
possible  to  use  established  firm-level  performance  information  to  make  decisions  about  the 
performance  of  individual  firm  processes  or  activities.  In  sum,  allocating  revenue  to  IT 
investments  allows  the  use  of  a  return-based  ratio,  both  within  a  firm’s  boundaries  and 
across  inter-firm  processes. 

Given  the  multiple  benefits  of  such  a  ROIT  ratio,  it  becomes  imperative  to  develop  a 
framework  for  ex-post  allocation  of  revenue  and  costs  to  IT  at  any  firm,  functional,  or  process 
level  to  map  the  economic  impact  of  IT  initiatives.  Since  IT  researchers  and  managers  have 
recently  come  under  a  great  deal  of  scrutiny  to  justify  the  role  of  IT  on  productivity,  it  is 
necessary  for  the  IS  community  to  build  a  common  framework  for  understanding,  evaluating, 
and  justifying  the  impact  of  IT  investments  on  process  improvement  and  firm  performance. 
For  IS  researchers,  such  a  ratio  would  largely  eliminate  the  need  to  infer  the  role  of  IT  using 
the  typical  approach  of  controlling  for  all  possible  inputs  and  assuming  that  the  residual  value 
is  due  to  IT.  This  ratio  would  permit  estimating  the  returns  of  specific  IT  initiatives  (e.g., 
customer  relationship  management,  enterprise  resource  planning,  e-commerce)  at  either  the 
firm  or  process  level. 

Managers  would  benefit  from  the  use  of  a  performance  metric  that  uniquely  specifies  the 
impact  of  IT  investments,  allowing  them  to  justify,  longitudinally  track,  and  finally  estimate  the 
impact  of  their  IT  investment  decisions  on  the  performance  of  specific  processes  at  any  level 
of  aggregation.  From  a  managerial  standpoint,  it  is  important  to  reiterate  that  the  proposed 
approach  is  based  on  historical  data,  which  are  commonly  used  in  the  finance  and 
accounting  literatures  to  describe  firm  performance  over  a  given  time.  This  historical  analysis 
also  is  used  to  predict  future  firm  performance.  Consequently,  we  do  not  make  any  claims 
about  prescribing  future  firm  or  process  output,  future  revenues,  customer  value,  or  ROIT, 
other  than  making  predictions  based  on  historical  data. 

The  paper’s  primary  contribution  is  to  introduce  the  possibility  of,  and  propose  a  conceptual 
framework  for,  allocating  revenue  to  IT  investments  and  estimating  a  return  on  IT 
investments.  Given  the  difficulty  in  developing  a  solid  theory  and  a  corresponding 
operationalization  for  unambiguously  allocating  revenue  across  the  firm,  the  primary  goal  of 
this  paper  is  to  stimulate  a  debate  and  future  research  on  measuring  the  returns  on  IT 
investment  through  the  allocation  of  revenue  to  productive  assets  within  the  firm.  Such 
debate  would  help  establish  superior  theories  and  operationalizations  for  allocating  revenue 
derived  from  IT  investments,  thereby  helping  managers,  analysts,  and  investors  predict  and 
assess  the  value-adding  role  of  such  investments. 

Section  2  reviews  prominent  current  approaches  for  measuring  ROIT,  concluding  with  a  set 
of  criteria  for  developing  a  new  measurement  approach.  Section  3  presents  and  describes 
the  proposed  knowledge-based  view  for  measuring  the  ROIT,  and  Section  4  presents  a  case 
example  that  demonstrates  the  operationalization  of  the  proposed  method.  Section  5 
discusses  the  implications  of  the  proposed  approach,  and  Section  6  concludes  by  urging  IS 
researchers  to  develop  their  own  approaches  for  allocating  revenue  and  enabling  new  forms 
of  IT  performance  measurement. 
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Review  of  Measuring  Returns  on  It  Investments 

Given  the  importance  of  establishing  and  measuring  the  returns  on  IT  investments,  the  IS 
and  related  literatures  have  proposed  numerous  methods.  This  brief  literature  review  section 
aims  only  to  give  a  representative  view  of  the  original  papers  in  order  to  develop  a  set  of 
criteria  for  a  new  approach  to  measure  ROIT.  For  a  more  comprehensive  review  of  the 
literature,  please  see  Kohli  and  Devaraj  (2003)  or  Melville,  Kraemer,  and  Gurbaxani  (2004). 

This  literature  review  will  demonstrate  that  there  are  four  key  requirements  for  developing  a 
defensible  approach  for  measuring  the  return  on  IT  investments: 

•  A  perspicuous  theoretical  framework 

•  Specific  allocation  of  revenue  and  cost  to  IT  initiative 

•  Mapping  of  the  IT  impact  at  any  level  of  aggregation  (firm  and  process) 

•  A  means  to  describe  all  outputs  in  common  units  of  measurement 

Research  on  estimating  the  value  added  by  IT  can  be  categorized  at  two  levels  of  analysis: 
(1)  corporate  (firm)  level,  and  (2)  sub-corporate  (process)  level.  We  summarize  the  focus  of 
Section  2  in  Table  1  and  review  the  studies  contained  therein  in  sections  2.1  and  2.2. 


Table  1  -  Common  Approaches  for  Measuring  the  Return  on  IT 


Level  of 
Analysis 

Approach 

Focus  Example 

Key  Key 

Assumption  Advantage 

Limitation 

Aggregate 
Corporate 
(firm)  level 

Process  of 
Elimination 

Treats  effect  of 

IT  on  ROI  as  a 
residual  after 
accounting  for 
other  capital 
investments 

Knowledge 

capital 

(Strassmann 
2000a,  b) 

ROI  difficult 
to  measure 
directly 

Uses 

commonly 

accepted 

financial 

analysis 

techniques 

and  existing 

accounting 

data 

Cannot  drill 
down  to 
effects  of 
specific  IT 
initiatives 

Production 

Theory 

Determines  the 
effects  of  IT 
through  input 
output  analysis 
using 

regression 

modeling 

techniques 

Brynjolfsson 
&  Hitt  (1996) 

Economic 
production 
function 
links  IT 
investment 
input  to 
productivity 
output 

Uses 

econometric 
analysis  on 
large  data 
sets  to  show 
contributions 
of  IT  at  firm 
level 

"Black-box" 
approach 
with  no 
intermediate 
mapping  of 
IT's 

contributions 
to  outputs 

Resource- 

Based 

View 

Linking  firm 
core  capabilities 
with 

competitiveness 

Jarvenpaa  & 

Leidner 

(1998) 

Uniqueness 
of  IT 

resource  = 

competitive 

advantage 

Strategic 
advantage 
approach  to 

IT  impacts 

Causal 
mapping 
between  IT 
investment 
and  firm 
competitive 
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advantage 
difficult  to 
establish 

Corporate/ 

sub¬ 

corporate 

Option 

Pricing 

Model 

Determines  the 
best  point  at 
which  to 
exercise  an 
option  to  invest 
in  IT 

Benaroch  & 

Kauffman 

(1999) 

Timing 
exercise 
option  = 
value 

Predicting 
the  future 
value  of  an 

IT 

investment 

No 

surrogate  for 
revenue  at 
sub¬ 
corporate 
level 

Sub¬ 

corporate 

(Process) 

Family  of 
Measures 

Measure 
multiple 
indicators  to 
derive  unique 
contributions  of 

IT  at  sub¬ 
corporate  level 

Balanced 
score-card 
(Kaplan  & 
Norton, 

1996) 

Need 
multiple 
indicators  to 

measure 

performance 

Captures 
complexity 
of  corporate 
performance 

No  common 
unit  of 
analysis/ 
theoretical 
framework 

Cost- 

Based 

Use  cost  to 
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Firm  Level  Approaches 

Firm  level  approaches  aim  to  understand  the  contribution  of  firm  assets  (e.g.,  knowledge  and 
technology)  to  a  firm’s  performance.  At  the  firm  level,  the  most  popular  techniques  are:  (a) 
process  of  elimination,  (b)  production  theory,  (c)  resource-based  view,  and  (d)  option  pricing, 
as  described  below: 

The  Process  of  Elimination  Approach 

In  the  process  of  elimination  approach,  once  the  capital  costs  (e.g.,  equipment,  IT  assets, 
real  estate)  are  accounted  for  and  income  is  proportionately  is  reduced,  the  residual  is 
asserted  to  be  revenue  attributed  to  knowledge  capital  (Strassman,  2000a,  2000b).  Baruch 
Lev’s  method  (www.stern.nyu.edu/~blev)  on  valuing  firm  knowledge  capital  follows  the 
process  of  elimination  approach.  This  method  estimates  the  value  of  knowledge  capital  by 
subtracting  the  expected  income  from  a  firm’s  tangible  and  financial  assets  from  past  and 
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expected  earnings  to  give  the  firm’s  “knowledge  earnings.”  Lev  applies  a  discount  rate  to  the 
average  post-tax  return  for  three  knowledge-intensive  industries  to  obtain  the  firm’s 
knowledge  capital.  In  essence,  this  approach  identifies  the  value  of  knowledge  capital  by 
subtracting  the  effect  of  all  other  assets  and  accounting  for  what  remains. 

A  similar  method  applies  to  IT:  since  costs  attributed  to  all  cost  categories  (except  IT)  reduce 
income,  the  remaining  income  is  attributed  to  IT.3  The  limitation  of  the  ‘process  of  elimination’ 
approach  is  that  it  only  applies  at  the  aggregate  firm  level,  and  thus  is  unlikely  to  allow 
precise  inferences  about  IT-driven  performance  improvements  at  the  process  level  (Birchard 
and  Nyberg,  2001). 

The  Production  Theory  Approach 

Others  have  used  economic-based  production  theory  to  determine  the  various  contributions 
of  inputs  to  the  firm’s  output.  Hitt  and  Brynjolfsson  (1996)  asessed  the  value  of  IT  in  terms  of 
productivity,  profitability,  and  consumer  welfare.  Extending  this  research,  Brynjolfsson  and 
Yang  (1996)  broadly  report  the  effect  of  IT  on  overall  firm  performance.  The  resulting 
“production  function”  (Brynjolfsson  and  Hitt,  1996,  p.  545)  can  be  modeled  using  economic 
theory  to  determine  the  unique  contributions  of  IT.  Computer  and  non-computer  capital,  IT 
labor,  and  other  labor  expenses  are  viewed  as  the  inputs  (which  represent  all  firm  spending 
and  investments)  and  output  in  terms  of  monetary  or  physical  units.  This  neoclassical 
economic  theory  of  production  treats  the  firm  as  a  “black  box”  and  attempts  to  deduce  the 
relationship  between  inputs  and  outputs  without  reference  to  activities  within  the  firm. 

The  strength  of  such  approaches  derives  from  their  reliance  on  commonly  accepted 
economic  theories  and  the  use  of  existing  accounting  data  that  makes  them  transparent  for 
review  and  comparison.  However,  various  criticisms  have  been  leveled  at  these  approaches, 
including  that  they  do  not  “adequately  control  for  other  factors  (other  than  IT)  that  drive  firm 
profits”  (Bharadwaj,  2000,  p.  170).  Similarly,  Im  et  al.  (2001,  104)  confirm,  “Because  many 
factors  influence  firm  performance,  it  is  difficult  to  establish  causality  between  IT  investments 
and  firm-level  output  performance.”  This  lack  of  intermediate  mapping  of  IT  impacts  on 
processes  makes  firm-level  approaches  problematic  for  providing  the  feedback  necessary  to 
help  managers  determine  whether  IT  investments  do  pay  off. 

The  Resource-Based  View  Approach 

The  resource-based  view  (RBV)  attempts  to  overcome  these  shortcomings  by  reformulating 
the  question  and  attempting  to  link  a  firm’s  output  to  firm-specific  IT  resources,  such  as  IT 
capabilities  (Bharadwaj,  2000).4  A  limitation  of  the  RBV  is  that  it  does  not  posit  a  common, 


3 

Strassman  (2000b)  provides  a  brief  review  of  the  essence  of  how  the  knowledge  capital  metric  is  a 
form  of  economic  value-added.  “By  filtering  out  the  contributions  of  financial  capital  from  the  reported 
profits,  we  are  left  with  a  residual  that  is  entirely  attributable  to  what  knowledge  capital  has  actually 
delivered...  As  a  reminder,  one  can  compute  the  knowledge  capital  of  a  firm  by  taking  a  year's  worth 
of  economic  value-added-the  financial  returns  from  assets  that  do  not  show  up  in  financial  reports.”  He 
also  reports  on  the  essence  of  Lev’s  proprietary  method  for  calculating  knowledge  capital:  “Lev 
introduces  a  simple  ratio  for  determining  the  valuation  of  a  firm's  knowledge  capital.  Strassman 
(2000a)  defines  it  as  "...the  normalized  earning  minus  earnings  from  tangible  and  financial  assets 
divided  by  knowledge  capital  discount  rate." 

4  Resources  include  financial  assets,  IT,  employees,  and  company  brand.  Capabilities  are  firm-specific 
and  refer  to  the  ability  to  leverage  resources  to  produce  value.  Jarvenpaa  and  Leidner  (1998,  p.  343) 
summarized  the  approach:  “Focusing  on  the  firm  level  analysis,  the  resource-based  view  emphasizes 
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granular  unit  of  analysis  that  would  allow  a  specific  mapping  between  a  firm’s  use  of  IT  and 
the  resulting  value-adding  performance.  Therefore,  it  would  still  be  difficult  to  definitively 
determine  the  specific  contribution  of  a  given  IT  initiative. 

The  Option  Pricing  Approach 

The  application  of  option  pricing  models  (OPM)  for  evaluating  IT  initiatives  has  attracted 
increasing  attention  (Benaroch  and  Kauffman,  1999). 5  Basically,  the  OPM  approach 
determines  the  best  point  at  which  to  exercise  an  option  to  invest.  In  the  context  of  real 
options  investments  six  variables  are  used  to  make  the  decision  (Amram  and  Kulatilaka, 
1999,  p.  37):  (1)  the  current  value  of  the  underlying  asset,  (2)  the  time  to  the  decision  date, 
(3)  the  investment  cost  or  exercise  price  (also  called  the  strike  price),  (4)  the  risk-free  rate  of 
interest,  (5)  the  volatility  of  the  underlying  asset  (which  is  often  the  only  estimated  output), 
and  (6)  cash  payouts  or  non-capital  gains  returns  to  holding  the  underlying  assets.  The  OPM 
approach  has  some  implicit  assumptions  that  are  potentially  limiting.  For  example,  net 
present  value  is  used  in  the  risk  calculation,  which  requires  an  assumption  about  projected 
cash  flows.  However,  there  is  no  cash  flow  directly  attributable  to  most  firm  processes 
because  their  outputs  are  not  salable  to  end  customers  (without  the  outputs  of  all  other 
processes),  limiting  the  OPM  applicability  at  the  process  level. 

The  current  aggregate  (firm)  level  approaches  do  not  appear  capable  of  precisely 
determining  the  role  of  IT  on  performance  at  the  process  level  so  as  to  benefit  managers  who 
must  implement  IT  initiatives  at  the  sub-corporate  level.  Such  process-level  approaches  are 
described  next: 

Process  (Sub-Corporate)  Level 

At  the  process  level,  approaches  to  determining  the  impact  of  IT  can  be  classified  as:  (a) 
family  of  measures,  (b)  cost-based,  and  (c)  knowledge  value-added. 

The  Family  of  Measures  Approach 

The  Family  of  Measures  ( Balanced  Scorecard  and  Intellectual  Capital  Navigator)  approaches 
advocate  the  need  to  measure  multiple  indicators  to  derive  unique  IT  contributions  at  the 
sub-firm  level.  The  Balanced  Scorecard  approach  uses  4-5  key  performance  indicators 
selected  by  management  to  determine  the  success  of  a  given  strategic  firm  thrust  (Kaplan 
and  Norton,  1996).  The  managerial  team  might  select  the  IT  initiative  for  assessment  via  a 
set  of  outcome  indicators  (customer  satisfaction,  financial  performance,  employee 
satisfaction,  process  performance).  Edvinsson  and  Malone’s  (1997)  Intellectual  Capital 
Navigator  allows  a  firm  to  identify  up  to  140  variables  (e.g.,  laptops/employee,  IT 
expense/employee,  IT  staff/staff,  IT  literacy  of  employees)  that  account  for  the  performance 
of  its  intangible  assets,  including  IT  initiatives  (p.  109).  The  limitation  of  the  Family  of 
Measures  approaches  is  the  lack  of  a  common  theoretical  framework  and  unit  of  analysis 
that  would  convincingly  link  IT  investments  to  a  firm’s  performance  (Bharadwaj,  2000), 
leading  to  an  inherent  subjectivity  problem  in  terms  of  specifying  the  weight  of  each  measure. 


the  resources  possessed,  developed,  and  deployed  by  an  organization  and  understanding  the 
relationship  of  those  internal  resources  with  performance  competitiveness.”  These  resources  and 
capabilities  are  difficult  to  copy,  and  they  can  thus  provide  a  basis  for  a  competitive  advantage. 

5  The  options  pricing  model  can  be  used  to  assess  returns  on  IT  investments  at  both  the  corporate  and 
process  level. 
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Cost-based  methods  often  use  underlying  replacement  costs  (e.g.,  transfer  pricing,  internal 
markets,  outsourcing)  to  determine  the  value  of  IT  (Housel  and  Bell,  2001).  These 
approaches  assume  that  the  cost  of  IT  is  in  some  way  proportionate  to  its  value.  For 
example,  the  cost  to  replace  or  outsource  IT  is  presumed  to  be  proportionate  to  the  value  it 
adds  to  process  performance.  Similar  cost-based  approaches  assume  that  by  introducing  a 
market  mechanism  where  firm  managers  submit  bids  for  IT  services,  the  resulting  internal 
“market  price”  is  representative  of  the  true  value  of  IT  (Ba,  Stallaert  and  Whinston,  2001). 

The  Activity  Based  Costing  Approach 

Activity  Based  Costing  (ABC)  is  a  common  cost-based  approach.  It  is  useful  for  finding  and 
evaluating  the  true  costs  of  process  activities  (Johnson  and  Kaplan,  1987).  Applications  of 
ABC  to  measuring  IT  impacts  assume  that  any  costs  saved  (or  processes  simplified,  and 
thus  costs  reduced)  by  the  IT  are  a  direct  reflection  of  its  value.  This  assumption  may  be  true 
in  cases  where  costs  are  reduced,  and  process  outputs  remain  constant  or  increase. 

The  conceptual  limitation  of  cost-based  approaches  is  the  lack  of  a  surrogate  for  revenue 
(Johnson,  1992).  The  problem  of  using  cost  as  a  surrogate  for  value  is  that  all  information  is 
contained  in  the  denominator  of  the  productivity  ratio.  Ideally,  the  information  about  value 
should  come  from  the  revenue  side  of  the  firm's  performance  (the  numerator). 

Other  Practical  Approaches 

The  current  economic  environment  has  also  placed  a  great  urgency  on  achieving  greater 
precision  and  providing  valid  and  reliable  approaches  for  estimating  the  ROIT.  For  example, 
the  practitioner  CFO  community  has  sought  measures  for  capturing  IT  initiatives.  Bannan 
(2001)  argued  that  CFOs  would  like  to  see  a  ROIT  measure,  but  since  such  a  measure  is 
lacking,  they  have  to  settle  for  less  concrete  and  more  general  measures  (e.g.,  number  of 
hits,  page  views).  Lenatti  (2003)  argued  that  a  way  to  estimate  the  ROI  on  IT  is  necessary  to 
secure  project  funding  and  measure  project  success. 

Conceptual  Framework 

A  General  Approach  for  Describing  Outputs  in  Equivalent  Units 

The  purpose  of  firms  is  to  produce  value  via  their  processes  by  transforming  inputs  (e.g., 
energy,  information)  into  outputs  (e.g.,  final  products/services).  The  proposed  conceptual 
framework  for  measuring  ROIT  is  based  on  a  method  to  describe  the  outputs  of  any  given 
process  in  equivalent  measurement  units.  Such  methods  would  be  able  to: 

•  Compare  all  processes  in  terms  of  their  relative  productivity 

•  Allocate  revenue  to  common  measurement  units  of  output 

•  Describe  the  value  added  by  IT  resources  in  terms  of  the  outputs  they  produce 

•  Relate  outputs  to  the  cost  to  produce  those  outputs  in  common  measurement 
units 

•  Describe  a  common  unit  of  measurement  for  firm  productivity 

The  Knowledge  Based  View 

The  knowledge-based  view  (KBV)  argues  that  the  firm’s  primary  function  (and  reason  for 
existence)  is  to  leverage  knowledge  into  productive  outcomes  (Kogut  and  Zander,  1996; 
Nahapiet  and  Ghoshal,  1998;  Nonaka,  1994;  Zander  and  Kogut,  1995).  The  KBV  describes 
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firm  resources  and  capabilities  as  knowledge  sets  (Leonard-Barton,  1992).  Knowledge  is  the 
stock  of  intellectual  assets  accumulated  through  experience,  learning,  and  ongoing  practices 
(Sambamurthy,  2000).  Capabilities  (the  effectiveness  in  executing  business  processes)  are 
thus  generated  through  an  ongoing  process  of  absorbing  information,  converting  it  into 
knowledge,  and  utilizing  knowledge  to  effectively  undertake  functional  activities.  Knowledge 
broadly  encompasses  facts,  symbols,  data,  discussions,  workflows,  tasks,  whiteboard 
sessions,  human  expertise,  and  scientific  understanding  (Becerra-Fernandez  and 
Sabherwal,  2001). 

In  order  to  fully  understand  what  is  knowledge,  it  is  important  to  distinguish  between 
knowledge  and  information  by  proposing  two  distinct  categories:  (a)  information  or  explicit, 
codifiable  knowledge  and  (b)  know-how  or  tacit  knowledge  (Alavi  and  Leidner,  2001;  Grant, 
1996;  Kogut  and  Zander,  1992;  Nonaka,  1991).  Information  or  explicit,  codifiable  knowledge 
can  be  easily  exchanged,  shared,  stored,  and  retrieved  without  much  loss.  On  the  other 
hand,  know-how  or  tacit  knowledge  is  the  information  that  has  been  processed  in  the  minds 
of  individuals  through  deliberation,  learning,  and  judgment  (Alavi,  2000).  Despite  this 
distinction,  it  is  important  to  clarify  that  these  two  categories  are  not  dichotomous,  but  they 
are  mutually-dependent  and  reinforcing  facets  of  knowledge  (Polanyi,  1975).  As  Tsoukas 
(1996)  suggests,  tacit  and  explicit  knowledge  are  inseparable.  Nonaka  and  Takeuchi  (1995) 
further  propose  that  knowledge  is  created  through  interactions  among  different  combinations 
of  tacit  and  explicit  knowledge.  Therefore,  the  proposed  view  encompasses  both  tacit  and 
explicit  knowledge.6 

A  Knowledge-Based  View  of  Business  Processes 

Our  proposed  theoretical  framework  is  based  on  the  amount  of  change  each  business 
process  (P)  produces,  which  is  essentially  the  difference  between  input  (A)  and  output  (B), 
as  shown  in  Figure  1. 


B 


INPUT 


PROCESS 


OUTPUT 


Figure  1.  Describing  Knowledge  Proportionately  to  Process  Change 


Following  KBV,  we  contend  that  the  change  between  a  certain  input  A  and  output  B  is  based 
on  the  knowledge  (tacit  and  explicit)  needed  to  drive  the  business  process  (P).  It  follows  that 
introducing  changes  to  a  process,  through  IT  for  example,  that  do  not  produce  changes  in 
the  process  output  (e.g.,  in  terms  of  its  characteristics,  cost  or  quality)  adds  no  value.  To 
illustrate,  if  a  process  is  fully  or  partially  automated  via  the  use  of  IT,  then  the  amount  of 
change  or  value  added  by  IT  can  be  measured  precisely  as  long  as  A  is  changed  into  B. 


6  Even  if  the  proposed  approach  encompasses  both  tacit  and  explicit  knowledge,  it  is  obviously  easier 
to  describe  explicit  knowledge  and  estimate  the  required  ‘learning  time’. 
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Conversely,  if  the  change  added  by  a  process  is  purely  dependent  on  the  way  the  process 
operates  (and  not  on  the  output  of  the  process),  then  every  process  change  would  represent 
a  change  in  value,  even  if  the  actual  output  of  the  process  does  not  change  at  all.  Therefore, 
the  description  of  a  process  in  terms  of  the  elementary  changes  it  requires  to  produce 
outputs  from  inputs  provides  a  common  units  surrogate  for  the  output.  In  this  way,  knowledge 
described  in  terms  of  the  changes  produced  by  a  process  can  become  a  common  units 
surrogate  for  the  output  of  the  process. 

a.  Revenue  Allocation 

At  a  given  point  in  time,  a  firm’s  revenue  is  best  represented  by  the  aggregate  output  of  its 
processes.  A  firm’s  revenue  is  obtained  simply  by  multiplying  the  price  it  charges  for  its 
products  by  the  quantity  sold.  It  is  important  to  mention  that  firms  price  their  products  to 
maximize  their  revenue,  and  price  is  often  independent  of  the  cost  needed  to  produce  the 
products.  Most  important,  it  is  beyond  the  scope  of  this  study  to  explain  or  predict  how 
revenue  is  received,  or  how  well  a  firm’s  products  are  priced  to  maximize  revenue.  Our  goal 
is  simply  to  take  historical  revenue  received  and  allocate  it  to  the  firm’s  productive  processes. 

Since  knowledge  essentially  drives  the  transformation  of  inputs  into  outputs  in  any  given 
process,  following  KBV,  the  total  knowledge  required  to  produce  this  aggregate  output  is  a 
surrogate  for  the  firm’s  total  revenue.  If  knowledge  can  be  described  in  common  units,  then  it 
is  possible  to  allocate  the  firm’s  revenue  to  these  units  of  knowledge.  This  would  allow  the 
establishment  of  a  common  price  per  unit  of  knowledge.  Thus,  it  follows  that  revenue  per  unit 
of  knowledge  is  a  surrogate  for  revenue  per  unit  of  common  output. 

Extended  to  IT  resources,  this  formulation  allows  a  direct  linkage  between  a  firm’s  revenue 
and  the  knowledge  distributed  among  its  IT  resources  (and  other  complementary  or 
independent  resources)  needed  to  produce  revenue.  It  then  becomes  possible  to  allocate  the 
revenue  produced  by  the  knowledge  in  firm  processes,  and  specifically  the  knowledge 
contained  in  the  IT  resources. 

IT  is  often  just  one  of  many  resources  within  a  firm,  and  problems  often  arise  because  it  often 
complements  other  firm  resources  in  generating  value  (Barua  and  Mukhopadhyay,  2000). 
Complementarity  represents  an  enhancement  of  resource  value  since  synergies  among  firm 
and  IT  resources  are  likely  to  create  greater  returns  (Milgrom  and  Roberts,  1995).  Following 
complementarity  theory,  Barua  et  al.  (1996)  proposed  the  “calculus  of  reengineering”  in 
which  IT  returns  may  be  a  non-linear  (super-modular)  function  of  the  contributing 
complementary  factors.  Consequently,  Barua,  Kriebel  and  Mukhopadhyay  (1995)  developed 
a  model  of  IT  value  in  which  the  impact  of  IT  on  firm  performance  is  mediated  by 
intermediate  processes. 

To  address  potential  complementarities  between  IT  and  other  synergistic  factors,  our 
proposed  method  allocates  revenue  to  a  joint  factor  combining  IT  and  complementary 
resources,  and  translates  it  into  common  units  of  knowledge.  This  formulation  captures  non¬ 
linear  synergistic  effects.  Having  captured  the  revenue  produced  by  the  joint  factor,  revenue 
can  then  be  allocated  among  the  constituent  factors  (i.e.,  IT  resources  and  other  synergistic 
resources).  This  allocation  can  be  theoretically  performed  at  any  level  of  analysis  by 
analyzing  the  details  of  each  process  and  specifying  the  contribution  of  each  productive 
factor  (knowledge  required  to  perform  each  factor’s  contribution). 
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A  trade-off  arises  between  the  level  of  detail  required  to  precisely  allocate  revenue  and  the 
effort  required  to  disentangle  and  thoroughly  describe  the  process  to  identify  each  unique 
contribution.  In  such  cases,  management  needs  to  step  in  to  allocate  revenue  based  on  a 
simple  heuristic  (roughly  equal  contribution),  or  to  specify  the  level  of  detail  in  the  process 
description  that  is  needed  to  make  a  more  accurate  allocation.7 

b.  Cost  Allocation 

The  proposed  approach  also  provides  a  means  to  estimate  the  denominator  (cost)  of  the 
productivity  ratio,  where  the  costs  of  routine  process  executions,  execution  errors,  poor 
quality,  lack  of  training,  and  poorly  designed  IT,  for  example,  can  be  captured  in  terms  of  the 
cost  to  execute  any  given  process.  Following  the  proposed  approach,  cost  is  also  allocated 
using  common  units  of  knowledge,  and  the  aggregate  cost  estimates  at  the  process  level  can 
thus  be  directly  linked  to  the  firm’s  total  cost.  Thus,  the  costs  are  comparable  across  the 
organization,  and  they  can  be  used  to  compare  the  cost  of  producing  process  outputs  across 
processes. 

Our  approach  is  similar  to  cost  accounting  methods  because  fixed  costs  are  uniformly  spread 
across  all  processes,  representing  a  constant  factor  in  calculating  the  productivity  (revenue 
over  cost)  across  firm  processes.  Labor  costs,  however,  are  usually  distinct  across 
processes,  particularly  IT  labor,  which  often  represents  one  of  the  highest  cost  drains. 

Allocating  costs  among  complementary  factors  is  problematic  as  well,  especially  for  IT 
resources  that  usually  create  value  only  in  combination  with  other  resources.  Our  cost 
allocation  approach  would  first  determine  the  total  cost  for  the  joint  (combinatory)  factor  in 
common  units,  then  for  each  of  the  constituent  factors.  The  granularity  and  accuracy  of  the 
cost  allocation  decision  is  based  on  the  degree  of  detail  desired  by  management. 

c.  A  Productivity  (Revenue  over  Cost)  Ratio 

Summarizing  the  preceding  discussion,  since  we  derive  revenue  and  cost  allocation 
independently,  our  approach  can  establish  a  productivity  ratio  (revenue  over  cost).  This  ratio 
allocates  a  percentage  of  revenue  to  a  given  process  based  on  (a)  the  amount  of  knowledge 
required  to  produce  the  process  outputs  in  the  numerator,  over  (b)  the  cost  to  employ  the 
knowledge  in  the  denominator. 

This  productivity  ratio  can  be  used  at  any  aggregation  level  (process  or  firm)  to  first  estimate 
the  Returns  on  Knowledge  (ROK)  for  each  chosen  level  of  analysis,  and  then  extend  this 
ratio  to  IT  resources  to  estimate  ROIT  investments. 

Operationalizing  Knowledge  with  “Learning  Time” 

Several  ways  to  operationalize  the  amount  of  process  change  have  been  proposed  in  the 
literature,  such  as  information  bits,  process  instructions,  Hay  knowledge  points,  and  Jackson 
Structural  diagrams)  (see  Housel  and  Bell  (2001)  fora  review). 

While  there  are  numerous  ways  to  estimate  the  amount  of  knowledge  required  in  a  process, 
following  the  KBV,  learning  time  is  proposed  to  be  a  simple,  quick,  and  convenient 


7  This  procedure  is  not  much  different  from  any  cost  accounting  allocation  decisions.  For  example,  when 
two  cost  units  share  a  similar  overhead,  a  management  decision  needs  to  be  made  in  terms  of  the 
percentage  that  needs  to  be  absorbed  for  each  unit. 
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operationalization  because  knowledge  contained  in  any  given  process  is  proportionate  to  the 
time  it  takes  to  learn  it.  Processes  with  predetermined  outputs  may  be  described  in  terms  of 
the  amount  of  time  it  takes  the  average  learner  to  learn  how  to  produce  these  outputs.  The 
proposed  learning  time  estimation  method  assumes  that  the  time  it  takes  for  an  "average 
person"  to  learn  to  execute  a  given  process  is  proportional  to  the  amount  of  knowledge 
needed  (Kanevsky  and  Housel,  1998)  and  is  a  descriptive  and  practical  surrogate  for  the 
corresponding  amount  of  process  change.  The  proposed  operationalization  is  summarized  in 
Table  1 . 


Table  2.  The  Operationalization  of  the  Knowledge-Based  Conceptualization 

1. 

If  A=B,  no  change  has  been  added  by  process  P 

2. 

If  A  is  changed  by  P  in  to  B,  then  “value”  is  proportionate  to  the  amount  of 
“change” 

3. 

Change  can  be  measured  by  the  amount  of  knowledge  needed  to  make  the 
changes 

4. 

Knowledge  is  proportional  to  the  time  it  takes  an  average  learner  to  acquire  the 
knowledge 

5. 

Value  and  change  are  then  proportionate  to  the  time  it  takes  an  average 
learner  to  acquire  the  knowledge  required  to  make  a  change 

Applied  to  IT  investments,  learning  time  is  the  time  an  average  learner  needs  to  acquire  the 
knowledge  needed  to  use  an  IT  system  to  drive  a  business  process  or  produce  the  process 
output. 

This  description  of  the  study’s  theoretical  framework  and  proposed  operationalization  can  be 
described  in  the  context  of  management  decision-making  with  simple  examples  (Table  3). 

The  learning  time  analogy  can  be  used  to  establish  an  analytic  measure  of  the  common  units 
of  change  executed  by  firm  processes  with  predetermined  outputs.5  Because  the  learning 
time  proxy  allows  for  the  measurement  of  process  changes  in  common  units,  it  then 
becomes  possible  to  allocate  revenue  in  proportion  to  the  amount  of  learning  time  at  any 
level  of  analysis. 


Table  3.  An  Illustrative  Example  of  the  Proposed  Operationalization 


Let's  assume  that  we  teach  the  "average"  person  everything  she  needs  to  know,  including 
how  to  produce  all  the  outputs  of  IT  investments  and  how  to  produce  all  outputs  for  any 
given  firm.  In  a  very  real  sense  then,  her  knowledge  of  the  firm  would  be  the  embodiment  of 
the  firm’s  value-added  processes.  Therefore,  it  is  these  core  processes  (e.g.,  selling, 
marketing,  production,  accounting,  finance)  that  change  process  inputs  to  value-added 
outputs.  When  combined,  these  outputs  generate  the  firm’s  revenue. 

We  can  put  this  understanding  to  the  test  with  a  simple  example.  In  the  widget  company, 


5  Some  processes  do  not  have  predetermined  outputs  such  as  those  that  are  highly  creative. 
However,  the  outputs  of  these  processes  eventually  find  their  way  into  those  processes  that  are  more 
deterministic  and  can  therefore  be  accounted  for  within  the  context  of  processes  with  predetermined 
outputs.  If  they  do  not,  then  they  are  in  “inventory”  for  possible  later  use  in  the  processes  that  produce 
company  products. 
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there  is  one  person,  the  owner,  who  makes  and  sells  widgets.  This  person  knows  all  there 
is  to  know  to  make  widgets,  which  sell  for  $1 .  The  owner's  sales-production  knowledge  can 
be  used  as  a  surrogate  for  the  $1  of  revenue  generated  by  his  application  of  the  core 
process  knowledge.  We  can  ascertain  how  long  it  would  take  the  widget  company  owner  to 
transfer  all  the  necessary  sales  and  production  knowledge  to  a  new  owner.  Furthermore, 
we  could  use  these  learning  times  to  allocate  the  $1  of  revenue  between  the  sales  and 
production  processes.  In  this  sense,  the  knowledge  is  a  surrogate  for  the  amount  of  change 
produced  by  the  sales  and  production  processes. 

For  simplicity,  let’s  assume  that  it  takes  100  hours  for  a  new  owner  to  learn  both  processes, 
with  70  hours  spent  to  learn  how  to  make  the  widget  and  30  hours  to  learn  how  to  sell  it.  Of 
the  70  hours  of  learning  time,  let’s  assume  that  20  hours  were  used  to  learn  how  to  produce 
the  outputs  of  the  IT  used  to  support  the  production  process.  This  would  indicate  that  70%  ol 
the  knowledge,  elementary  changes  or  complexity,  and  value  added  were  contained  in  the 
production  process  and  30%  in  the  sales  process.  It  would  follow  that  $.70  of  the  revenue 
was  generated  by  the  production  knowledge,  and  $.30  by  the  sales  knowledge. 

Having  determined  how  much  it  will  cost  to  use  the  sales  and  production  knowledge,  we 
would  then  have  a  ratio  of  knowledge  revenue  to  knowledge  cost,  or  Return  on  Knowledge 
(ROK).  It  is  a  simple  extrapolation  from  there  to  generate  the  ROIT  ratio  by  partitioning  the 
amount  of  knowledge  the  IT  used  to  produce  the  outputs  of  these  two  processes.  Then,  by 
allocating  revenue  to  these  IT  outputs  and  subtracting  the  cost  to  produce  these  IT  outputs 
(divided  by  the  cost  to  produce  the  IT  outputs),  we  would  have  an  ROIT  estimate. 

Let's  assume  that  the  total  cost  to  sell  and  produce  the  widget  was  $.50:  $.25  cost  for  sales 
and  $.25  for  production.  Of  the  production  cost,  $.05  was  the  cost  to  use  the  IT  supporting 
the  production  process.  We  would  conclude  that  the  production  process  provided  a  better 
utilization  of  the  knowledge  asset  (ROK=.70/.25=280%)  than  the  sales  process 
(ROK=.30/.25=120%).  Further  assume  that  the  IT  in  the  production  process  accounted  for 
20  units  of  output  and  cost  $.05  to  produce.  Thus  the  ROIT  would  be  .20-. 05/. 05  or  300%. 

The  revenue  attributable  specifically  to  the  knowledge  embedded  in  IT  and  the  cost  to  use 
IT  would  provide  the  ROIT  within  and  among  processes.  This  can  be  quite  revealing  in  that 
"all  IS  are  not  created  equal."  Some  highly  automated  processes  would  likely  provide  much 
lower  ROIT  compared  to  others,  where  there  is  a  lower  percentage  of  automation  but  where 
the  IS  provides  much  more  "bang  for  buck." _ 


Reliability  and  Accuracy  of  ‘Learning  Time’  Operationalization 

Due  to  concerns  over  the  accuracy  of  the  learning  time  estimates,  multiple  learning  time 
methods  have  been  used  to  determine  the  level  of  correlation  among  estimates.  Such 
estimates  can  be  checked  against  standard  training  times  for  given  process  tasks  (Section 
4).  For  example,  subject  matter  experts  (SME)  are  asked  to  estimate  how  long  it  would  take 
a  common  reference  point  learner  to  learn  how  to  produce  the  process  outputs.  In  this  case, 
they  are  also  asked  to  assume  that  the  supporting  IT  process  has  been  removed,  and  that 
the  common  reference  point  learner  must  now  learn  how  to  produce  the  IT  outputs.  Where 
possible,  these  estimates  are  then  compared  to  actual  training  times  for  learning  how  to 
produce  specific  process  outputs.  The  assumption  is  that  there  is  an  average  learning  time 
across  many  learners.  As  a  practical  exercise,  learning  time  estimates  for  common  reference 
point  learners  can  be  grounded  so  that  any  biases  will  be  equally  distributed. 
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When  SME  have  trouble  estimating  learning  times,  the  process  instruction  method  is  applied, 
and  the  SME  are  asked  to  describe  the  instructions  required  to  teach  the  common  reference 
point  learner  how  to  produce  the  outputs  of  a  process.  If  estimation  problems  still  remain 
after  these  efforts,  a  more  detailed  and  time-consumer  analysis  is  performed  for  the  problem 
areas.  In  general,  normal  sampling  methods  apply  to  all  approaches  in  order  to  ensure  a 
reasonable  estimate  of  the  change  produced  by  the  focal  processes. 

Validating  ‘Learning  Time’  Estimates 

The  learning  time  method  can  also  be  complemented  by  the  process  instruction  or  binary 
query  approach  to  assess  the  reliability  of  the  ‘learning  time’  estimates.  The  learning  time 
approach  is  practical  and  enables  less  costly,  more  rapid  estimations  of  the  amount  of 
change  produced  by  processes.  Poorly  performing  processes  can  then  be  selected  for  more 
precise  analysis  techniques  as  required  to  support  resource  allocation  decisions. 

Finally,  the  proposed  learning  time  method  can  be  compared  with  other  estimates  of  process 
change  (e.g.,  process  instructions)  to  assess  the  reliability  of  the  obtained  estimates.  In  the 
learning  time  approach,  the  total  amount  of  learning  time  required  to  market,  finance,  sell, 
produce,  account  for,  and  distribute  a  firm's  outputs  (i.e.,  sellable  products  or  services)  is  a 
surrogate  for  the  revenue  derived  from  a  firm’s  outputs  during  a  given  sample  period.  The 
outputs  of  all  the  firm  processes  used  to  generate  this  revenue,  at  a  given  point  in  time,  can 
be  described  in  common  units  of  learning  time.  It  follows  that  “price  per  unit  of  output,”  or  its 
surrogate  “price  per  unit  of  knowledge,”  (which  is  derived  by  dividing  firm  revenue  by  the  total 
number  of  units  of  knowledge)  is  a  constant.  However,  the  cost  per  unit  of  knowledge  will 
vary  depending  on  the  cost  to  use  the  knowledge  (e.g.,  human  and  IT  resources)  to  produce 
a  process  output. 


Case  Example  Of  Learning  Time  Operationalization 

To  illustrate  the  use  of  the  learning  time  operationalization  for  measuring  the  value  added  by 
IT  investments,  we  present  an  example  derived  from  a  case  study  we  conducted  with  SBC 
Telecom.8 

This  example  demonstrates  the  use  of  normalized  and  actual  learning  time  for  estimating 
process  changes  in  equivalent  units  of  knowledge,  thus  allowing  revenue  allocation  at  the 
process  level  (and  to  the  supporting  IT).9 


8  Special  thanks  to  Professor  Richard  Chase  (University  of  Southern  California)  for  helping  us  develop 
this  case  example  to  help  explain  the  essence  of  the  proposed  operationalization  from  a  management 
perspective.  In  addition  to  SBC  Telecom,  the  proposed  ‘learning  time’  operationalization  has  been 
applied  in  over  100  companies  (both  profit  and  not  for  profit)  and  repeatedly  in  a  number  of  these 
companies.  Management  and  process  subject  matter  experts  found  the  learning  time  approach  to  be 
intuitive  and  relatively  easy  to  apply  to  derive  reliable  estimates.  A  frequent  comment  by  the 
participants  was  that  this  method  took  significantly  less  time  (on  average  14  days  for  one  analyst  to 
complete  an  analysis  of  a  core  process),  than  other  competing  methods  (e.g.,  activity-based  costing). 

9  A  number  of  compromises  have  been  made  for  the  sake  of  simplicity.  For  example,  in  the  aggregate 
level  example,  it  would  have  been  preferable  to  observe  the  actual  number  of  times  the  knowledge  in 
each  process  was  used  to  produce  outputs  within  a  given  sampling  period.  The  process  level  example 
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The  data-gathering  team  calculated  the  learning  times,  as  well  as  the  actual  number  of 
training  days  required  to  learn  all  the  sub-processes  of  the  firm’s  core  processes  for  the 
aggregate  level  example.  The  team  used  a  single  point  of  reference  for  learning  time 
estimates  (i.e.,  one  of  the  team  members)  to  ensure  that  biases  would  be  evenly  distributed 
across  all  estimates.  In  addition,  the  relative  learning  times  were  based  on  the  amount  of 
time  it  would  take  the  single  point  of  reference  learner  to  learn  all  the  processes  if  they  only 
had  a  total  of  100  months.  The  normalization  to  100-months  technique  has  been  used  to 
benchmark  the  telecom  industry  and  other  industry  segments,  including  the  consulting 
industry  (Housel  and  Horn,  1999). 

Case  Example — Correlations  among  Learning  Time  Estimates 

The  team  correlated  the  multiple  estimates  with  each  other  as  a  basic  estimate  of  their 
reliability.  Given  that  the  estimates  are  derived  using  a  common  theoretical  framework,  a 
simple  correlation  is  a  reasonable  approximation  of  the  reliability.  In  this  case  example,  the 
relative  learning  time  estimates  were  highly  correlated  with  actual  training  time  estimates 
(94%)  and  thus  with  the  training  time  estimates  being  used  for  all  subsequent  calculations 
(Table  4). 


Table  4.  Simple  Correlation  Table  for  Corporate  Level  Example 

Column  1 

Column  2 

Column  3 

Multiple  Knowledge  Estimates 

Relative  Learning  Time 

Actual  Average  Training 

Reliability 

(100  months) 

Period  (Hours) 

Marketing 

6 

500 

Ordering 

12 

923 

Provisioning 

36 

13,000 

Maintenance 

20 

7,300 

Billing 

7 

300 

Customer  Care 

11 

1,218 

Administrative  Support 

4 

1,000 

Sales 

4 

2,448 

Correlations 

Column  2  &  3=  94% 

The  purpose  of  getting  two  ( normalized  and  actual)  estimates  of  learning  time  was  to  check 
the  reliability  of  the  obtained  estimates.  In  other  instances  where  such  actual  learning  time 
referents  may  not  be  available,  SME  can  be  asked  to  assess  the  actual  learning  time,  and 
inter-rater  reliability  measures  can  then  be  taken.  SME  could  also  be  asked  to  decompose 
core  processes  into  their  sub-processes  and  then  estimate  how  long  it  would  take  the 
reference  point  learner  to  learn  how  to  produce  the  outputs  of  each  sub-process  step.  While 
greater  accuracy  requires  a  longer  time  commitment,  for  rough  estimates,  the  normalized 
and  actual  (via  estimates  among  several  SME  or  actual  training  times)  learning  time 
estimates  suffice.  A  correlation  of  .80  or  higher  is  typically  sufficient  to  proceed  with  the 
calculations  of  the  ROIT  estimates. 


provides  the  number  of  times  knowledge  was  used,  in  addition  to  the  learning  time  and  process 
instructions  estimates  for  change  in  sub-processes. 
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Case  Example — Interpretation  of  the  Return  on  Knowledge  (ROK)  Results 

The  calculations  of  the  ROK  ratios  are  included  in  Tables  5  and  6.  We  purposefully  left  out 
the  fixed  costs  because  they  were  spread  evenly  across  all  processes  (similar  to  traditional 
cost  accounting  methods).  For  example,  the  cost  for  power,  maintenance,  and  real  estate 
was  spread  evenly  across  all  processes.  The  most  significant  variable  cost  was  labor. 

The  Provisioning  and  Maintenance  processes  were  orders  of  magnitude  more  complex  than 
other  processes  and  required  much  more  time  to  learn  as  well  as  more  support  from  IT.  This 
is  because  these  two  processes  represent  an  aggregation  of  many  sub-processes  (for 
example,  in  the  case  example,  there  are  15  legacy  systems  required  to  provision  an  order, 
and  11  systems  to  complete  a  maintenance  order).  These  IT  systems  are  essentially  used  to 
manipulate  and  keep  inventory  records.  Much  less  time  is  required  to  learn  how  to  order, 
support,  sell,  market,  and  bill.  This  is  because  the  product  was  well  known  within  a  highly 
regulated  industry  where  the  customer  had  limited  choice. 


Table  5.  ROK  Estimates  for  Aggregate  Example 


Column  1 
Core 

Processes 

Column  2 
Learning 
Time 

Column  3 
IT 

Learning 

Time 

Column  4 
Total 
Learning 
(in  hours) 
(Col.  2  + 
Col  3) 

Column  5 
%  Total  LT 

(Col.  4/Total 
LT) 

Column  6 
Total  Annual 
Cost  Per 
Process 
Area 

Column  7 
Revenue 
Allocation 
(Col.  5*  Total 
Revenue) 

Column  8 
ROK 

(Col.  7 /Col. 
6 

■100%) 

Marketing 

500 

150 

650 

1.5% 

$2,700,000 

$2,350,585 

-13% 

Ordering 

923 

692 

1615 

3.75% 

2,875,000 

5,841,205 

103% 

Provisioning 

13000 

7800 

20800 

48.24% 

12,583,721 

75,218,734 

498% 

Maintenance 

7300 

4380 

11680 

27.09% 

10,016,279 

42,238,212 

322% 

Billing 

300 

240 

540 

1.25% 

4,025,000 

1,952,794 

-51% 

Customer 

Care 

1218 

853 

2071 

4.8% 

4,775,000 

7,487,880 

57% 

Corporate 

1000 

600 

1600 

3.71% 

6,425,000 

5,786,056 

-10% 

Sales 

2448 

1714 

4162 

9.65% 

20,000,000 

15,049,533 

-25% 

TOTALS 

26689 

16429 

43118 

100.0% 

63,400,000 

155,925,000 

146% 

Our  goal  was  to  generate  relative  return  performance  estimates.  Table  5  shows  the  ROK 
results,  which  provided  a  framework  for  beginning  the  process  of  prioritizing  IT  initiatives  to 
support  redesigned  processes.  The  ROK  performance  data  serve  the  purpose  of  providing  a 
baseline  from  which  to  iterate  various  process  redesign  models.  For  example,  the  ROIT  to 
support  the  sales  process  (Table  6)  provides  a  baseline  comparison  for  a  process  redesign 
using  a  new  customer  relationship  management  (CRM)  system  to  support  the  sales  process. 

One  method  for  estimating  the  learning  time  for  IT  is  to  ask  a  process  SME  to  estimate  how 
long  it  would  take  the  average  learner  to  learn  how  to  produce  the  output  of  the  IT,  if  the  IT 
failed.  Given  that  most  processes  have  not  always  been  automated  with  supporting  IT,  the 
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SME  would  estimate  how  long  it  would  take  to  teach  the  referent  learner  how  to  produce  the 
outputs  that  automation  is  now  producing.  In  case  the  IT  produces  new  outputs  that  were 
never  produced  manually,  the  SME  would  have  to  estimate  how  long  it  would  take  a  referent 
learner  to  learn  how  to  produce  the  new  outputs.  In  either  case,  the  SME  can  be  asked  to 
decompose  the  IT  outputs  into  the  steps  required  to  produce  the  IT  outputs  manually,  and 
these  steps  can  then  be  calibrated  in  terms  of  learning  time. 


Table  6.  ROK  Estimates  with  IT  Partitioned  from  Total  Learning  Time 


Column  1 

Core  Process 

Column  2 
IT  LT 

Column  3 
%  IT  LT  of  overall 
Total  LT 

(Col.  2/Total  IT  LT) 

Column  4 

IT  Costs 

Column  5 
Revenue  IT  LT 
(Col.  3*  Total 
Revenue) 

Column  6 
ROK  on  IT 
(Col.  5/Col.  4)  - 
100% 

Marketing 

150 

.91% 

$600,0,00 

$540,978 

-10% 

Ordering 

692 

4.21% 

1,000,000 

2,495,711 

150% 

Provisioning 

7800 

47.48% 

3,583,720 

27,130,848 

685% 

Maintenance 

4380 

26.66% 

1016279 

15,796,553 

1454% 

Billing 

240 

1 .46% 

2,900,000 

865,565 

-70% 

Customer 

Care 

853 

5.19% 

2,000,000 

3,076,361 

54% 

Corporate 

600 

3.65% 

800,000 

2,163,911 

170% 

Sales 

1714 

10.43% 

2,000,000 

6,181,573 

209% 

TOTALS 

16429 

100% 

13,900,000 

59,251,500 

326% 

Interpretation  of  the  Return  on  IT  (ROIT)  Results 

Our  aggregate  level  analysis  included  the  amount  of  learning  time  that  was  attributable  to  the 
IT  to  support  the  core  processes.  Because  firm  management  had  developed  contingency 
scenarios  where  all  IT  systems  failed,  it  had  purposefully  developed  training  to  manually 
produce  the  same  outputs  as  the  IT  systems.  We  used  the  firm’s  training  time  estimates  for 
the  IT  learning  times,  which  permitted  partitioning  of  the  estimates  for  supporting  IT  (Table 
6)- 

This  partitioning  made  it  clear  that  some  forms  of  IT  support  provided  better  returns  than 
others.  For  example,  the  IT  system  that  supported  the  sales  process  provided  substantially 
better  returns  than  the  IT  system  that  supported  the  billing  process,  and  the  maintenance 
process  resulted  in  the  best  ROIT.  Even  though  the  legacy  IT  systems  supporting 
provisioning  and  maintenance  were  older  file  processing  systems,  they  had  been  specifically 
created  to  support  these  highly  optimized  core  processes.  However,  in  the  case  of  the  highly 
automated  legacy  billing  systems,  the  outputs  of  the  legacy  system  process  cost  more  than 
the  actual  revenue  allocated  to  the  process.  This  result  indicated  that  billing  was  a  process  in 
need  of  serious  attention  by  management,  and  led  them  to  place  it  on  a  watch  list  for  future 
reengineering  with  creation  of  a  new  billing  module  explicitly  designed  to  be  implemented 
with  a  sales  and  services  CRM  system. 

The  poor  performance  of  the  marketing  IT  was  not  a  primary  concern  of  the  firm’s 
management  team  because  it  was  believed  that  this  area  would  not  benefit  from  further 
automation.  Some  areas  may  be  more  prone  to  benefit  from  IT  support  than  others.  It  is  also 
possible  that  the  marketing  IT  may  not  have  been  utilized  to  its  fullest  capacity  because  the 
marketing  personnel  were  not  familiar  with  its  advanced  statistical  modeling  and  simulation 
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capabilities.  Because  our  approach  only  measures  IT  systems  in  use  and  not  IT  systems  in 
inventory,  it  can  only  indicate  the  performance  of  IT  in  actual  use.  There  are  numerous 
examples  of  IT  that  has  much  greater  functionality  than  is  being  utilized  in  the  processes 
supported  by  the  IT.  A  poor  ROIT  may  indicate  that  the  functionality  of  the  IT  is  under¬ 
utilized,  the  legacy  IT  requires  expensive  maintenance,  or  the  IT  system  is  used  infrequently. 

This  example  may  appear  counter-intuitive  because  the  legacy  IT  systems  were  so 
productive.  However,  the  results  indicate  that  even  legacy  IT  systems  could  provide  an 
acceptable  return  when  deployed  in  processes  for  which  they  have  been  specifically 
designed  and  optimized  for  a  specific  purpose.  It  follows  that  pure  reliance  on  the  type  of  IT 
(e.g.,  file  processing  system  -  legacy  systems,  web-based)  may  not  be  the  critical 
differentiator  in  terms  of  predicting  IT  performance.  Process  design  may  be  the  most  crucial 
issue  in  predicting  and  maintaining  the  highest  ROIT. 

This  case  example  therefore  demonstrates  that  the  cost  of  IT  investments  may  not 
necessarily  be  related  to  the  output  (value)  it  produces.  For  example,  the  IT  maintenance 
system  provided  substantially  higher  returns  that  any  of  the  other  IT  systems.  This  result  also 
demonstrates  that  it  is  critical  to  derive  both  the  numerator  and  denominator  of  any  IT 
investment  from  different  sources:  namely,  the  numerator  for  revenue  allocation  and  the 
denominator  for  cost  allocation. 

In  summary,  this  case  example  demonstrates  that  our  approach  for  measuring  change  in 
processes  to  estimate  output  can  be  operationalized  and  measured  in  relatively  practical 
ways.  The  advantage  of  this  approach  is  that,  while  grounded  in  a  solid  theoretical 
framework,  it  can  easily  be  applied  in  practice.  This  approach  can  obtain  practical  estimates 
that  are  grounded  in  common  units  of  knowledge,  and  these  units  can  be  used  as  a 
surrogate  for  process  change.  Finally,  the  proposed  approach  can  be  readily  used  to  obtain 
estimates  for  the  specific  contribution  of  IT. 

A  Simplified  Approach  for  obtaining  Learning  Times  Estimates 

The  detailed  analysis  outlined  in  the  case  example  is  relatively  time  consuming;  however, 
managers  can  obtain  rough  estimates  by  conducting  a  quicker  analysis,  as  summarized  in 
Table  7. 


Table  7.  A  Quick  Analysis  of  the  SBC  Telecom  Case  Study 


To  conduct  a  quick  analysis  of  the  SBC  Telecom  company,  we  would  gather  the  various 
managers  of  core  processes  (sales,  marketing,  network  provisioning,  maintenance,  etc.). 
We  would  then  ask  them  to  estimate  how  long  it  would  take  the  ‘average’  learner  to  learn 
how  to  produce  the  output  of  each  core  area  with  the  following  boundary  condition:  we 
only  have  a  total  of  100  months  for  our  average  person  to  learn  everything  necessary  to 
generate  the  annual  revenue  at  SBC  Telecom.  This  form  of  normalization  would  lead  to 
quick,  rough  cut  estimates  of  the  distribution  of  outputs  among  the  various  core  areas, 
processes,  and  functions,  which  can  then  be  used  to  allocate  the  annual  revenue. 

We  can  also  ask  the  managers  to  estimate  the  percentage  at  which  an  area  is 
automated.  This  provides  a  way  to  estimate  the  allocation  of  revenue  to  IT.  Admittedly, 
this  is  a  very  rough  estimate,  but  it  provides  the  benefit  of  getting  the  managers  to  agree 
on  the  basic  principle  that  knowledge  can  be  used  to  describe  firm  outputs. 
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We  would  not  ask  the  executives  to  make  estimates  of  the  value  of  their  core  processes, 
since  this  might  degenerate  into  a  no-win  ‘dog’  fight.  In  contrast,  we  would  ask  them  to 
achieve  consensus  estimates  of  approximately  how  much  time  our  average  person 
should  use  (of  the  total  allotted  100  months),  to  learn  each  core  process,  including  how 
to  produce  the  outputs  of  IT  investments.  These  estimates  would  be  weighted  by  the 
number  of  employees  in  each  core  process.  This  is  a  rough  cut  way  of  estimating  how 
often  the  knowledge  in  a  given  process  is  used  annually.  The  percentage  of  knowledge 
for  each  process  (including  its  supporting  IT)  would  then  be  calculated  (process 
knowledge/total  amount  of  knowledge),  and  revenue  would  be  allocated  proportionately. 

If  we  wanted  to  understand  the  contribution  of  IT,  the  revenue  for  each  process  could  be 
further  partitioned  into  the  amount  attributable  to  the  knowledge  embedded  in  the 
supporting  IT.  The  annual  budget  for  each  area  can  be  used  to  estimate  the  cost  to  use 
the  given  core  process  knowledge  (in  most  high-tech  firms,  as  SBC  Telecom,  this  will 
simply  be  the  cost  for  employee  salaries  and  IT  costs).  The  final  step  would  be  to  divide 
the  allocated  revenue  by  the  cost  per  core  process  to  determine  the  relative  returns. 


Discussion 

Despite  numerous  approaches  to  assess  the  performance  impact  of  IT  investments,  the 
literature  has  not  conclusively  shown  that  IT  investments  have  a  positive  effect  on  either  firm 
or  process  performance.  To  address  this  paradox,  this  study  argues  that  a  common 
measurement  unit  is  required  to  allocate  the  revenue  and  cost  of  IT  investments  at  any  level 
of  analysis  (firm  and  process).  Following  the  KBV,  our  proposed  approach  (which  applies  to 
virtually  all  levels  of  analysis)  first  captures  the  knowledge  needed  to  drive  firm  processes 
(and  thus  change  of  inputs  into  outputs)  and  translates  it  into  a  numerical  form  that  allows 
allocation  of  revenue  in  proportion  to  the  value  added  by  this  knowledge.  Second,  it  also 
assesses  the  actual  cost  of  the  knowledge  needed  to  execute  the  given  process.  Given  that 
revenue  and  cost  are  independently  assessed,  their  resulting  ratio  provides  a  ROK  ratio.  This 
ratio  readily  applies  to  IT  initiatives  since  IT  is  an  integral  part  of  the  knowledge  needed  to 
drive  firm  processes.  Therefore,  a  Return  on  Information  Technology  (ROIT)  ratio  can  be 
derived. 

In  terms  of  the  operationalization  of  the  knowledge  required  to  drive  firm  processes,  we 
propose  to  employ  the  time  it  would  take  the  average  learner  to  acquire  that  knowledge 
needed  to  produce  a  process  output.  Average  learning  time  is  a  simple,  convenient,  and 
reliable  operationalization  for  capturing  the  complexity  of  the  knowledge  required  to  drive 
business  processes. 

Implications  for  Theory  and  Research 

There  are  several  implications  for  theory  and  research  of  employing  the  proposed  approach 
for  obtaining  the  ROK  and  ROIT  at  various  levels  of  analysis.  The  following  sub-sections 
describe  how  the  proposed  method  relates,  draws  upon,  and  extends:  (1)  the  IT  assessment 
literature  and  existing  methods  for  measuring  the  returns  on  IT  (e.g.,  option  pricing,  internal 
markets,  and  RBV),  and  (2)  the  theory  of  complementarities. 
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Implications  for  Existing  Approaches  for  Measuring  the  Return  on  IT 

As  reviewed  earlier,  many  approaches  have  been  proposed  in  the  IS  and  related  literatures 
to  measure  the  ROIT.  Our  method  could  complement  many  of  these  approaches,  such  as 
the  option  pricing  model,  the  internal  markets  model,  and  the  RBV,  as  described  below. 

Option  pricing  models  hold  great  promise  for  predicting  future  investments  in  IT,  but  they 
could  benefit  from  having  a  surrogate  for  cash-flow  and  discounted  cash-flow  at  the  process 
level,  such  as  the  ones  derived  from  our  approach.  The  historical  data  captured  over  time 
using  this  approach  would  also  provide  the  volatility  ranges  that  can  be  used  as  inputs  to 
option  pricing  models.  The  advantage  of  integrating  our  proposed  approach  with  existing 
option  pricing  models  is  that  this  new  source  of  raw  data  would  make  it  possible  to  evaluate 
options  based  on  comparable  data  (similar  to  financial  options  models).  This  would  provide 
researchers  the  necessary  historical  data  that  would  allow  them  to  generate  a  more 
comprehensive  application  of  option  pricing  models  at  the  process  level  in  terms  of  hedging, 
financial  risk,  and  other  historical  trends. 

Internal  market  models  would  also  benefit  from  the  ability  to  allocate  and  project  revenue  to 
various  IT  initiatives  at  the  process,  department,  or  firm  level.  This  would  allow  comparisons 
among  the  various  cost-based  and  knowledge-based  frameworks  for  predicting  how 
managers  would  negotiate  with  each  other  on  the  basis  of  projected  cash-flows  derived  from 
investments  in  new  IT  systems  at  the  process,  department,  or  firm  level. 

Finally,  RBV  methods  would  benefit  from  tracking  the  effects  of  specific  capabilities, 
deployed  in  people  as  well  as  IT.  This  could  be  achieved  by  identifying  the  capabilities  that 
produce  the  greatest  returns  over  time.  For  example,  if  resources  -  such  as  customer 
knowledge  that  supports  web-based  interfaces  -  did  result  in  a  competitive  advantage,  this 
should  also  be  reflected  through  the  contributions  of  such  capabilities  to  a  firm’s  performance 
over  time. 

Implications  for  the  Theory  of  Complementarities 

IT-enabled  business  processes  usually  include  the  deployment  of  both  IT  and  other 
complementary  resources,  since  synergies  among  IT  and  related  resources  are  likely  to 
render  greater  returns  due  to  potential  complementarities  (Milgrom  and  Roberts,  1995). 
Following  complementarity  theory,  Barua  et  al.  (1996)  argue  that  IT  returns  may  have  a  non¬ 
linear  (super-modular)  function  of  the  contributing  complementary  factors  that  is  higher  than 
the  sum  of  the  returns  of  individual  constituent  factors.  Following  the  theory  of  IT 
complementarities,  our  proposed  approach  does  not  assume  that  the  output  of  firm 
processes  is  a  linear  function  of  the  returns  to  IT  and  other  complementary  factors,  but  rather 
it  argues  that  any  complex  IT-related  processes  can  be  captured  as  long  as  their  output  can 
be  described  in  terms  of  the  knowledge  required  to  produce  the  output.  The  more  complex 
the  complementarities  between  IT  and  other  resources,  the  more  learning  time  it  would 
probably  take  to  master  the  knowledge  to  produce  such  outputs.  Having  identified  the  joint 
contribution  of  a  set  of  complementary  factors,  the  proposed  approach  theoretically  enables 
the  identification  of  each  factor’s  unique  contribution  by  analyzing  the  complex  process  in 
greater  detail.  Even  if  it  is  practically  superior  to  employ  simple  revenue  allocation  heuristics 
(e.g.,  equal  contribution)  without  scrutinizing  each  factor’s  distinct  contribution,  it  is 
theoretically  feasible  to  specify  the  unique  contribution  of  IT  and  other  complementary  factors 
at  virtually  any  level  of  analysis  and  specificity. 
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The  proposed  method  thus  has  implications  for  the  theory  of  IT  complementarities  in  the 
sense  that  it  first  identifies  the  contribution  of  any  complementary  effects,  and  it  can  then 
specify  each  factor’s  unique  contribution  by  describing  the  underlying  process  at  any  level  of 
disaggregation.  At  a  minimum,  this  approach  provides  one  possible  method  for  measuring 
the  complementary  effects  of  IT  resources  as  they  participate  in  the  production  of  complex 
process  outputs.  Most  important,  it  can  even  further  distinguish  the  unique  contributions  of  IT 
resources  by  describing  business  processes  at  a  greater  level  of  detail. 

Our  approach  also  assumes  that  one  of  the  advantages  of  having  a  firm  entity  is  derived 
from  the  complementarities  among  its  processes.  Using  this  approach,  process  managers 
will  quickly  recognize  that  any  benefits  obtained  at  the  expense  of  lowering  the  performance 
of  other  processes  would  only  be  temporary.  This  is  because  the  overall  firm  performance  is 
not  likely  to  improve  if  the  costs  are  pushed  from  one  area  to  another.  Therefore,  if  various 
complementary  processes  cooperate  to  provide  a  greater  overall  value,  the  result  will  be  a 
higher  revenue  to  allocate  to  all  processes  and  a  lower  cost  to  each  of  them  (and  thus  higher 
firm  performance). 

Implications  for  Practice 

The  proposed  approach  addresses  a  long-held  need  recognized  by  executives,  IT  managers, 
and  investors  -  how  to  leverage  and  measure  the  knowledge  embedded  in  IT  systems, 
employees,  and  related  productive  assets.10  By  applying  this  approach  specifically  to  IT 
resources,  managers  can  benefit  from  a  performance  metric  that  uniquely  specifies  the 
impact  of  IT,  allowing  them  to  justify,  track,  and  finally  assess  the  impact  of  their  IT 
investment  decisions  on  the  performance  of  specific  processes,  functions,  or  the  entire  firm. 
Moreover,  by  tracking  the  conversion  of  knowledge  into  process  value,  managers  can 
increase  the  productivity  of  their  business  processes. 

Managers  who  redesign  business  processes  require  a  method  for  determining  how  much 
their  process  design  decisions  will  influence  existing  processes,  at  first,  and  firm 
performance,  overall  (El  Sawy,  2001).  The  proposed  approach  provides  a  simple  and  reliable 
way  to  estimate  the  returns  that  alternative  process  designs  can  generate. 

Our  method  also  applies  to  cost  allocation  decisions,  allowing  firms  to  allocate  the  costs  of 
virtually  any  process  using  common  units.  In  doing  so,  it  enables  managers  to  compare  the 
cost  of  knowledge  across  processes,  allowing  them  to  identify  unreasonably  costly 
processes.  For  example,  Johnson  Controls  used  our  approach  to  arrive  at  a  more 
comparable  costing  methodology  for  two  products  coming  off  the  same  assembly  line.  One 
of  the  products  was  significantly  more  expensive,  but  when  it  was  decomposed  via  the 
knowledge  required  to  complete  the  production  process,  they  Johnson  Controls  discovered 
that  on  a  common  unit  of  output  basis  it  was  actually  much  less  expensive  to  produce. 

With  a  common  reference  point  to  discuss  the  revenue  and  cost  of  a  given  business  process, 
managers  can  focus  their  attention  on  improving  both  halves  of  the  return  equation  (revenue 
and  cost)  and  avoid  a  reliance  on  cost  as  the  sole  determinant  of  their  decision  making.  This 
dual  focus  also  provides  better  protection  for  investors,  who  ultimately  want  to  see  increasing 


1 0 

The  proposed  method  is  embedded  in  several  software  programs,  such  as  the  enterprise 
management  and  monitoring  software  suite  produced  by  eCGSoft,  Inc.  (www.ecqsoft.com).  This 
software  allows  ongoing  monitoring  of  knowledge  data,  which  is  a  relatively  routine  task  for  managers. 
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IT  returns.  In  fact,  investors  would  be  able  to  accurately  track  the  performance  of  IT 
investments  over  time  (without  having  to  disentangle  the  role  of  IT  investments),  allowing 
them  to  reward  firms  that  make  sound  IT  investments  that  improve  business  processes  and 
facilitate  future  cash  flows  rather  than  forcing  them  to  rely  on  alternative  means,  such  as 
announcements  regarding  IT  initiatives  to  infer  future  firm  performance  (Rodgers  and  Housel, 
2001). 

Limitations  and  Suggestions  for  Future  Research 

The  proposed  approach  has  a  number  of  limitations  that  call  for  future  research 
opportunities. 

First,  this  approach  applies  primarily  to  processes  with  predetermined  outputs,  assuming  that 
there  is  a  “shortest  description”  of  the  knowledge  required  to  change  inputs  into  outputs. 
However,  since  there  is  theoretically  no  shortest  description,  this  approach  requires  a 
compromise  for  inferring  the  shortest  process  description.  Moreover,  describing  the 
knowledge  required  to  drive  complex  processes  is  not  always  an  easy  task.  Accordingly, 
estimating  the  average  learning  time  for  producing  a  given  output  is  often  difficult  and  time- 
consuming,  especially  for  complex  processes.  Future  research  could  simplify  the  necessary 
description  of  knowledge-intensive  processes  and  find  alternative  operationalizations  that  are 
more  appropriate  for  complex  processes. 

Second,  our  method  may  not  readily  apply  to  creative  and  unpredictable  processes,  such  as 
R&D.  Nevertheless,  for  the  output  of  R&D  processes  to  be  of  value  to  firms,  it  must 
eventually  find  its  way  into  processes  with  predetermined  outputs,  such  as  manufacturing, 
production,  and  quality  control.  From  this  perspective,  it  is  possible  to  use  the  approach  to 
track  the  conversion  of  such  creative  outputs  into  value  as  they  are  embedded  in  processes 
with  predetermined  outputs.  However,  future  research  could  attempt  to  extend  the  proposed 
approach  to  creative  and  unpredictable  processes. 

Third,  the  proposed  method  is  an  analytic  technique  that  uses  historical  data,  similar  to  most 
cost  accounting  approaches.  Therefore,  it  is  not  explicitly  designed  for  making  prospective 
estimates.  In  fact,  most  methods  for  predicting  the  performance  of  a  firm  are  based  on 
extrapolations  of  historical  data.  However,  predicting  changes  in  revenue,  especially  in  a 
competitive  fashion,  is  a  natural  next  step  for  future  research.  Future  research  could  thus 
attempt  to  extend  the  proposed  method  to  make  prospective  estimates. 

Fourth,  establishing  a  market  price  for  the  output  of  non-profit  firms  has  been  a  problem  for 
public  economics  for  some  time.  One  valuation  approach  that  shows  promise  is  the  market 
comparables  method.  Following  this  method,  it  may  be  possible  to  establish  a  defensible 
theoretical  basis  for  comparing  non-profit  processes  with  their  for-profit  comparable 
processes.  Given  that  the  output  of  each  process  can  be  described  in  common  measurement 
units,  it  may  be  possible  to  assign  the  comparable  market  price  per  unit  from  the  for-profit 
firms  to  the  non-profit  ones.  Future  research  could  clarify  the  theoretical  soundness  and 
practical  operationalization  of  this  possible  extension. 

Finally,  similar  to  the  limitations  of  complementarity  theory,  it  is  not  easy  to  trace  the  exact 
contribution  of  each  complementary  factor.  Consequently,  since  IT  resources  are  often 
intertwined  with  other  complementary  factors,  it  may  not  be  possible  to  accurately  calculate 
the  ROIT,  especially  for  complex  processes.  Future  research  could  attempt  to  trace  the  exact 
contribution  of  the  underlying  complementary  factors. 
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Conclusion 

Assessing  the  performance  impact  of  IT  investments  is  one  of  the  holy  grails  of  the  IS  field. 
In  fact,  IT  executives  have  recently  come  under  a  great  deal  of  scrutiny  to  show  the  impact  of 
IT  on  firm  productivity  and  profitability.  It  is  thus  necessary  for  the  IS  field  to  build  a  common 
framework  for  understanding,  evaluating,  and  justifying  the  impact  of  IT  investments  on 
process  improvement  and  firm  performance.  Moreover,  it  is  incumbent  upon  IS  researchers 
to  join  their  colleagues  in  finance  and  accounting  to  develop  a  common  set  of  metrics  for 
assessing  the  performance  of  IT  investments  at  virtually  any  level  of  analysis. 

Toward  this  goal,  the  first  and  most  difficult  step  is  to  assess  how  existing  revenue  can  be 
allocated.  This  paper’s  primary  goal  is  to  stimulate  a  debate  and  future  research  on 
measuring  the  returns  on  IT  investment  through  the  allocation  of  revenue  to  productive 
assets.  Given  the  inherent  difficulty  in  developing  a  solid  theory  and  a  corresponding 
operationalization  for  unambiguously  allocating  revenue  across  the  firm,  this  paper  aims  to 
offer  a  preliminary  set  of  guidelines  for  formulating  the  problem  of  allocating  revenue  at  any 
level  of  analysis.  In  doing  so,  it  aims  to  entice  future  research  to  seek  answers  in  terms  of 
how  IT-driven  revenue  can  be  accurately  traced  back  to  its  origins,  hoping  to  stimulate  the 
development  of  new  approaches  to  solve  the  more  general  problem  of  estimating  and 
convincingly  proving  the  return  of  IT  investments. 

The  first  debate  will  focus  on  the  conceptual  underpinnings  of  how  value  can  be  traced  back 
to  its  origins,  whether  this  is  at  the  process,  function  or  firm  level.  The  proposed  KBV  theory 
basis  and  other  theoretical  lenses  could  be  debated  for  their  superiority  in  conceptualizing 
the  performance  impacts  of  IT  investments  at  various  levels  of  analysis.  While  the  proposed 
KBV  approach  is  uniform  for  all  levels  of  analysis,  it  may  be  possible  that  different  theories 
may  be  more  appropriate  for  different  levels  of  analysis.  A  second  debate  could  revolve 
around  the  operationalization  of  the  various  theories  for  revenue  allocation.  These  two 
interrelated  debates  would  help  establish  superior  theories  and  operationalizations  for 
allocating  revenue  due  to  IT  investments,  thereby  helping  managers,  analysts,  and  investors 
to  predict  and  assess  the  value-potential  of  IT. 


References 

Alavi,  M.  (2000)  “Managing  Organizational  Knowledge,”  In  R.  W.  Zmud  (Ed.),  Framing  the 
Domains  of  IT  Management.  Cincinnati,  OH:  Pinnaflex. 

Alavi,  M.  and  D.E.  Leidner  (2001)  “Knowledge  Management  and  Knowledge  Management 
Systems:  Conceptual  Foundations  and  Research  Issues,”  MIS  Quarterly  (25)  1,  pp.  107- 
136. 

Amram,  M.  and  N.  Kulatilaka  (1999)  Real  Options:  Managing  Strategic  Investment  in  an 
Uncertain  World.  Boston:  Harvard  Business  School  Press. 

Ba,  S.,  J.  Stallaert,  and  A.  Whinston  (2003)  “Optimal  Investment  in  Knowledge  within  a  Firm 
Using  a  Market  Mechanism,”  Management  Science  (12)1,  pp.  200-214. 

Barua,  A.,  H.C.  Kriebel  and  T.  Mukhopadhyay  (1995)  "Information  Technologies  and 

Business  Value:  An  Analytic  and  Empirical  Investigation,"  Information  Systems  Research 
(6)1,  pp.  3-23. 

Barua,  A.,  S.  Lee,  and  A.  Whinston  (1996)  "The  Calculus  of  Reengineering,"  Information 
Systems  Research  (7)4,  pp.  409-428. 


222  Journal  of  the  Association  for  Information  Systems,  Vol.  6,  No.  7,  pp.  1 99-226/July  2005 


Pavlou  et  al. /Measuring  the  Return  on  IT 


Barua,  A.  and  T.  Mukhopadhyay  (2000)  “Information  Technology  and  Business  Performance: 
Past,  Present,  and  Future,”  In  R.  W.  Zmud  (Ed.),  Framing  the  Domains  of  IT 
Management.  Cincinnati,  OH:  Pinnaflex. 

Bannan,  K.  (2001)  “Measure  for  Measure,”  e-CFO,  Special  Summer  Issue,  pp.  58-59. 
Becerra-Fernandez,  I.  and  R.  Sabherwal  (2001)  “Organizational  Knowledge 
Management:  A  Contingency  Perspective,”  Journal  of  Management  Information  Systems 
(18)1 ,  pp.  23-55. 

Benaroch,  M.  and  R.  Kauffman  (1999  )  “A  Case  for  Using  Real  Options  Pricing  Analysis  to 
Evaluate  Information  Technology  Project  Investments,”  Information  Systems  Research, 
(10)1,  pp.  70-86. 

Birchard,  B.  and  A.  Nyberg  (2001)  “On  Further  Reflection:  Do  EVA  and  Other  Value  Metrics 
Still  Offer  a  Good  Mirror  of  Company  Performance?”  CFO  Magazine,  March  1 ,  2001 . 

Bharadwaj,  A.  (2000)  “A  Resource-Based  Perspective  on  Information  Technology  Capability 
and  Firm  Performance:  An  Empirical  Investigation,”  MIS  Quarterly,  (24)1,  pp.  169-196. 

Brynjolfsson,  E.  and  L.  Hitt  (1996)  “Paradox  Lost?  Firm-level  Evidence  on  the  Returns  on 
Information  Systems  Spending,”  Management  Science,  (42)4,  pp.  541-558. 

Brynjolfsson,  E.  and  L.  Hitt  (1998)  “Beyond  the  Productivity  Paradox,”  Communications  of  the 
ACM,  (41)8,  pp.  49-55. 

Brynjolfsson,  E.  and  S.  Yang  (1996)  “Information  Technology  and  Productivity:  A  Review  of 
the  Literature,”  Advances  in  Computers,  (43)2,  pp.  179-214. 

Chappell,  A.  (1999)  “Is  Knowledge  Transfer  the  Key  to  Retention?  “  The  Club  Insider,  May, 

pp.  16-20. 

Cover,  T.  and  J.  Thomas  (1991)  Elements  of  Information  Theory.  New  York:  John  Wiley  & 
Sons. 

Devaraj,  S.  and  R.  Kohli  (2003)  "Performance  Impacts  of  Information  Technology:  Is  Actual 
Usage  the  Missing  Link?"  Management  Science  (49)3,  pp.  273-289. 

Dos  Santos,  B.,  K.  Peffers  and  D.C.  Mauer,  (1993)  “The  Impact  of  Information  Technology 
Investment  Announcements  on  the  Market  Value  of  the  Firm,”  Information  Systems 
Research  (4)1,  pp.  1-23. 

Edvinsson,  L.  and  M.  Malone  (1997)  Intellectual  Capital.  New  York:  Harper  Business. 

El  Sawy,  O.  (2001)  Redesigning  Enterprise  Processes  for  e-Business.  Boston:  McGraw-Hill. 

El  Sawy,  O.,  A.  Malhotra,  S.  Gosain,  and  K.  Young  (1999)  “IT-Intensive  Value  Innovation  in 
the  Electronic  Economy:  Insights  from  Marshall  Industries,”  MIS  Quarterly,  (23)3,  pp. 
305-335. 

Grant,  R.  M.  (1996)  “Toward  a  Knowledge  Based  Theory  of  the  Firm,”  Strategic  Management 
Journal  (17),  pp.  109-122. 

Hitt,  L.  and  E.  Brynjolfsson  (1996)  “Productivity,  Business  Profitability  and  Consumer 
Surplus:  Three  Different  Measures  of  Information  Technology  Value,”  MIS  Quarterly, 
(20)2,  pp.  144-162. 

Housel,  T.  and  A.  Bell  (2001)  Managing  and  Measuring  Knowledge.  Boston:  McGraw-Hill. 

Housel,  T.,  O.  El  Sawy,  J.  Zhong,  and  W.  Rodgers  (2001)  “Measuring  the  Return  on 

Knowledge  Embedded  in  Information  Technology,  ’’Proceedings  of  the  21st  International 
Conference  on  Information  Systems,  New  Orleans,  LA,  pp.  16-19. 

Housel,  T.  and  S.  Horn  (1999)  “Knowledge  Management  for  the  Telecommunications 
Industry,”  Professional  Education  International,  Inc.,  International  Engineering 
Consortium,  Chicago,  IL. 

Housel,  T.  and  V.  Kanevsky  (1995)  “Re-Engineering  Business  Processes:  A  Complexity 
Theory  Approach  to  Value  Added,”  INFOR,  (33)4,  pp.  248-262. 

Im,  K.,  K.  Dow,  and  V.  Grover  (2001)  “Research  Report:  A  Reexamination  of  IT  Investment 
and  the  Market  Value  of  the  Firm  -  An  Event  Study  Methodology,”  Information  Systems 
Research  { 12)1,  pp.  103-117. 


Journal  of  the  Association  for  Information  Systems,  Vol.  6,  No.  7,  pp.  199-226/July  2005 


223 


Pavlou  et  al. /Measuring  the  Return  on  IT 


Jarvenpaa,  S.  and  D.  Leidner  (1998)  “An  Information  Company  in  Mexico:  Extending  the 
Resource-based  View  of  the  Firm  to  a  Developing  Country  Context,”  Information  Systems 
Research,  (4)9,  pp.  342-361. 

Johnson,  H.  (1992)  Relevance  Regained:  From  Top-Down  to  Bottom-Up  Employment.  New 
York:  The  Free  Press. 

Johnson,  H.  and  R.  Kaplan  (1987)  Relevance  Lost:  The  Rise  and  fall  of  Management 
Accounting.  Boston:  Harvard  Business  School  Press. 

Kanevsky,  V.  and  T.  Housel  (1998)  “The  Learning-Knowledge-Value  Cycle"  in  Knowing  in 
Firms:  Understanding,  Managing,  and  Measuring  Knowledge.  G.  Von  Krogh,  J.  Roos, 
and  Kleine,  D.  (Eds),  New  York:  Sage. 

Kaplan,  R.  and  D.  Norton  (1996)  The  Balanced  Scorecard.  Boston:  Harvard  Business  School 
Press. 

Kogut,  B.  and  U.  Zander  (1992)  “Knowledge  of  the  Firm,  Combinative  Capabilities,  and  the 
Replication  of  Technology,”  Organization  Science  (3)3,  pp.  383-397. 

Kogut,  B.  and  U.  Zander  (1996)  “What  do  Firms  do?  Coordination,  Identity,  and  Learning,” 
Organization  Science  (7),  pp.  502-518. 

Kohli,  R.,  and  S.  Devaraj  (2003)  “Measuring  Information  Technology  Payoff:  A  Meta-Analysis 
of  Structural  Variables  in  Firm-Level  Empirical  Research,”  Information  Systems  Research 
(14)2,  pp.  127-145. 

Lenatti,  C.  (2003)  “Grinding  Away  on  ROI,”  CFO -IT,  (8)  19,  pp.  22-29. 

Leonard-Barton,  D.  (1992)  “Core  Capabilities  and  Core  Rigidities:  A  Paradox  in  Managing 
New  Product  Development,”  Strategic  Management  Journal  (13)  1 ,  pp.  1 11-125. 

Melville,  N.,  K.  Kraemer,  and  V.  Gurbaxani  (2004)  “Review:  Information  Technology  and 
Organizational  Performance:  An  Integrative  Model  of  IT  Business  Value,”  MIS  Quarterly 
(28)2,  pp.  283-322. 

Mendelson,  H.  (2000)  "Organizational  Architecture  and  Success  in  the  Information 
Technology  Industry,"  Management  Science  (46)4,  pp.  513-529. 

Milgrom,  P.,  Y.  Qian  and  J.  Roberts  (1991)  "Complementarities,  Momentum,  and  the 
Evolution  of  Modern  Manufacturing,"  American  Economic  Review  (81)2,  pp.  84-88. 

Milgrom,  P.  and  E.B.  Roberts  (1995)  "Complementarities  and  Fit:  Strategy,  Structure,  and 
Organizational  Change  in  Manufacturing,"  Journal  of  Accounting  and  Economics  (19)2, 
pp.  179-208. 

Nahapiet,  J.  and  S.  Ghoshal  (1998)  “Social  Capital,  Intellectual  Capital,  and  the 

Organizational  Advantage,”  Academy  of  Management  Review  (23) 2,  pp.  242-266. 

Nonaka,  I.  (1991)  “The  Knowledge-Creating  Company,”  Harvard  Business  Review,  (69),  pp. 
96-104. 

Nonaka,  I.  (1994)  “A  Dynamic  Theory  of  Organizational  Knowledge  Creation,”  Organization 
Science  (5)1,  pp.  14-37. 

Nonaka,  I.  and  H.  Takeuchi  (1995)  “The  Knowledge-Creating  Company,”  New  York:  Oxford 
University  Press. 

Polanyi,  M.  (1975)  “Personal  Knowledge,”  In  M.  Polanyi  &  H.  Prosch  (Eds.),  Meaning, 
Chicago:  University  of  Chicago  Press,  pp.  22-45. 

Ray,  G.,  W.A.  Muhanna,  and  J.B.  Barney  (2005)  “Information  Technology  and  the 
Performance  of  the  Customer  Service  Process:  A  Resource-Based  Analysis,”  MIS 
Quarterly,  (forthcoming). 

Richardson,  V.  and  R.W.  Zmud  (2002)  “Systematic  Investor  Biases  Regarding  Stock  Market 
Returns  Following  Information  Technology  Expenditures,”  working  paper,  University  of 
Oklahoma. 

Rodgers,  W.  and  T.  Housel  (2001)  “Investment  Decision  Making  in  the  High  Technology 
Industry:  Implications  for  the  Accounting  Profession,”  Proceedings  of  the  European 
Accounting  Association  Annual  Congress,  pp.  456-471 . 


224  Journal  of  the  Association  for  Information  Systems,  Vol.  6,  No.  1,  pp.  1 99-226/July  2005 


Pavlou  et  al. /Measuring  the  Return  on  IT 


Sambamurthy,  V.  (2000)  “Business  Strategy  in  Hyper  competitive  Environments:  Rethinking 
the  Logic  of  IT  Differentiation,”  In  R.  W.  Zmud  (Ed.),  Framing  the  Domains  of  IT 
Management.  Cincinnati,  OH:  Pinnaflex. 

Strassman,  P.  (1997)  The  Squandered  Computer.  New  Haven:  The  Information  Economics 
Press. 

Strassman,  P.  (2000a)  “Accumulating  Knowledge  Efficiency,”  Knowledge  Management, 
February,  http://www.kmmaq.com. 

Strassman,  P.  (2000b)  “In  Search  of  Knowledge  Capital”.  Knowledge  Management, 
September,  http://www.kmmaq.com. 

Subramani,  M.R.  and  E.  Walden  (2002)  “The  Impact  of  E-commerce  Announcements  on  the 
Market  Value  of  Firms,”  Information  Systems  Research,  (12)2,  pp.  135-154. 

Tsoukas,  J.  (1996)  “The  Firm  as  a  Distributed  Knowledge  System:  A  Constructionist 
Approach,”  Strategic  Management  Journal  (17)1,  pp.  11-25. 

Zander,  U.  and  Kogut,  B.  (1995)  “Knowledge  and  the  Speed  of  Transfer  and  Imitation  of 
Organizational  Capabilities,”  Organization  Science  (6)1,  pp.  76-92. 


About  the  authors 

Paul  A.  Pavlou  is  an  Assistant  Professor  of  Information  Systems  at  the  University  of 
California  at  Riverside.  He  received  his  Ph.D.  from  the  University  of  Southern  California  in 
2004.  His  research  focuses  on  information  systems  strategy  in  turbulent  environments, 
electronic  commerce,  and  institutional  trust  building.  His  research  has  appeared  in  MIS 
Quarterly,  Information  Systems  Research,  the  Journal  of  the  Academy  of  Marketing  Science, 
the  International  Journal  of  Electronic  Commerce,  the  Journal  of  Strategic  Information 
Systems,  the  Proceedings  of  the  ICIS  Conference,  and  the  Best  Paper  Proceedings  of  the 
Academy  of  Management  Conference.  Paul  received  the  MIS  Quarterly  ‘Reviewer  of  the 
Year’  award  for  2003.  He  also  received  the  ‘Best  Doctoral  Dissertation  Award’  of  the  2004 
International  Conference  on  Information  Systems  (ICIS). 

Tom  Housel  specializes  in  valuing  intellectual  capital,  telecommunications,  information 
technology,  value-based  business  process  reengineering,  and  knowledge  value 
measurement.  He  is  currently  a  tenured  Full  Professor  for  the  Information  Sciences 
(Systems)  Department.  His  areas  of  teaching  include:  information  technology  for  homeland 
defense,  decision  support  systems,  knowledge  management,  electronic  business, 
telecommunications,  and  reengineering.  His  current  research  focuses  on  the  use  of  “Real 
Options”  models  in  identifying,  valuing,  maintaining,  and  exercising  options  in  military 
decision  making.  Prior  to  joining  NPS,  he  also  was  a  Research  Fellow  for  the  Center  for 
Telecommunications  Management  and  Associate  Professor  at  the  Marshall  School  of 
Business  at  the  University  of  Southern  California.  Tom  has  been  the  Chief  Business  Process 
Engineer  for  Pacific  Bell,  where  he  completed  numerous  reengineering  projects  and 
developed  a  new  objective  method  for  measuring  the  value-added  by  reengineering.  His  last 
assignment  in  the  corporate  world  was  as  the  Chief  of  Consumer  Market  Research  for 
Telecom  Italia  in  Venice,  Italy  where  he  developed  new  methods  for  predicting  the  adoption 
rates  for  new  interactive  multimedia  broadband  applications.  He  is  Managing  Partner  for 
Business  Process  Auditors,  a  firm  that  specializes  in  training  Big  Six  consultants,  large 
manufacturing  and  service  companies  in  the  Knowledge  Value-Added  methodology  for 
objectively  measuring  the  return  generated  by  corporate  knowledge  assets/intellectual 
capital.  He  received  his  Ph.D.  from  the  University  of  Utah  in  1980.  He  won  the  prestigious 
Society  for  Information  Management  award  for  best  paper  in  the  field  in  1986.  His  work  on 
measuring  the  value  of  intellectual  capital  has  been  featured  in  a  Fortune  cover  story 


Journal  of  the  Association  for  Information  Systems,  Vol.  6,  No.  7,  pp.  199-226/July  2005 


225 


Pavlou  et  al. /Measuring  the  Return  on  IT 


(October  3,  1994)  and  Investor’s  Business  Daily,  numerous  books,  professional  periodicals, 
and  academic  journals  (most  recently  in  the  Journal  of  Intellectual  Capital  vol  2  2005).  His 
latest  books  include:  “Measuring  and  Managing  Knowledge”  and  “Global 
Telecommunications  Revolution:  The  Business  Perspective”  with  McGraw-Hill  (both  in  2001). 

Waymond  Rodgers  is  a  C.P.A.  and  Professor  and  in  the  Graduate  School  of  Management 
at  the  University  of  California  in  Riverside.  His  degrees  are  from  the  University  of  Southern 
California,  Ph.D.  in  accounting  information  systems,  and  an  experimental  psychology  post¬ 
doctorate  from  the  University  of  Michigan.  His  experiences  include  working  as  an  auditor  with 
Ernst  &  Young  and  PriceWaterhouseCoopers,  as  well  as  a  commercial  loan  officer  with 
Union  Bank.  His  primary  research  areas  are  auditing,  commercial  lending  decisions,  decision 
modeling,  ethics,  trust  issues,  intellectual  capital,  and  knowledge  management.  Professor 
Rodgers’  has  published  in  the  Communication  of  the  ACM,  European  Accounting  Review, 
Journal  of  Business  Ethics,  Journal  of  Applied  Social  Psychology,  Journal  of  Economic 
Psychology,  Management  Science,  among  other  journals.  Finally,  he  is  the  recipient  of  major 
research  grants  from  the  Ford  Foundation,  National  Institute  of  Health,  National  Science 
Foundation,  Department  of  Defense,  and  the  Navy  Personnel  Research  and  Development 
Center. 

Erik  Jansen  has  a  Ph.D.  from  the  University  of  Southern  California  with  emphases  in 
organizational  behavior,  strategic  reward  systems  and  inquiring  systems.  He  is  a  Senior 
Lecturer  in  the  Department  of  Information  Sciences  and  an  affiliate  of  the  Center  on 
Terrorism  and  Irregular  Warfare  at  the  Naval  Postgraduate  School.  Dr.  Jansen’s  private  and 
public  sector  research  has  focused  on  issues  of  organizational  design,  strategic  human 
resource  systems,  and  motivation  and  careers,  primarily  in  the  contexts  of  innovation  and 
collaboration.  His  current  research  focuses  on  knowledge  flows  and  collaboration  in  the 
context  of  national  security  and  defense.  He  currently  teaches  “Organizational  Design  for 
Special  Operations,"  “Organizational  Structures  and  Systems,”  and  “Modern  Philosophy  of 
Science”.  His  articles  have  appeared  in  the  Academy  of  Management  Review,  Recent 
Advances  in  Organizational  Development,  The  Journal  of  Business  Venturing,  and 
Entrepreneurship,  Theory  and  Practice. 

Copyright  ©  2005  by  the  Association  for  Information  Systems.  Permission  to  make  digital 
or  hard  copies  of  all  or  part  of  this  work  for  personal  or  classroom  use  is  granted  without  fee 
provided  that  copies  are  not  made  or  distributed  for  profit  or  commercial  advantage  and  that 
copies  bear  this  notice  and  full  citation  on  the  first  page.  Copyright  for  components  of  this 
work  owned  by  others  than  the  Association  for  Information  Systems  must  be  honored. 
Abstracting  with  credit  is  permitted.  To  copy  otherwise,  to  republish,  to  post  on  servers,  or  to 
redistribute  to  lists  requires  prior  specific  permission  and/or  fee.  Request  permission  to 
publish  from:  AIS  Administrative  Office,  PO  Box  2712  Atlanta,  GA,  30301-2712,  Attn: 
Reprints,  or  via  e-mail  from  ais@aisnet.org. 


226  Journal  of  the  Association  for  Information  Systems,  Vol.  6,  No.  7,  pp.  1 99-226/July  2005 


ourAnal  of  the 

ssociation  for 


EDITOR 


Information 


k-7y  stems 

ISSN:  "l 536-9323 


Sirkka  L.  Jarvenpaa 
University  of  Texas  at  Austin 


JAIS  SENIOR  EDITORS 


Soon  Ang 

Nanyang  Technological  University 

Izak  Benbasat 

University  of  British  Columbia 

Matthias  Jarke 

Technical  University  of  Aachen 

Kalle  Lyytinen 

Case  Western  Reserve  University 

Tridas  Mukhopadhyay 

Carnegie  Mellon  University 

Robert  Zmud 

University  of  Oklahoma 

JAIS  EDITORIA 

L  BOARD 

Ritu  Agarwal 

University  of  Maryland 

Paul  Alpar 

University  of  Marburg 

Anandhi  S.  Bharadwaj 
Emory  University 

Yolande  E.  Chan 

Queen’s  University 

Alok  R.  Chaturvedi 

Purdue  University 

Roger  H.L.  Chiang 

University  of  Cincinnati 

Wynne  Chin 

University  of  Houston 

Ellen  Christiaanse 

University  of  Amsterdam 

Alan  Dennis 

Indiana  University 

Amitava  Dutta 

George  Mason  University 

Robert  Fichman 

Boston  College 

Henrique  Freitas 
Universidade  Federal  do 

Rio  Grande  do  Sul 

Guy  G.  Gable 

Queensland  University  of 
Technology 

Rudy  Hirschheim 

Louisiana  State  University 

Juhani  livari 

University  of  Oulu 

Matthew  R.  Jones 

University  of  Cambridge 

Elena  Karahanna 

University  of  Georgia 

Robert  J.  Kauffman 

University  of  Minnesota 

Prabhudev  Konana 
University  of  Texas  at 

Austin 

Kai  H.  Lim 

City  University  of  Hong 

Kong 

Claudia  Loebbecke 
University  of  Cologne 

Mats  Lundeberg 

Stockholm  School  of  Economics 

Stuart  E.  Madnick 
Massachusetts  Institute  of 
Technology 

Ann  Majchrzak 

University  of  Southern 
California 

Ryutaro  Manabe 

Bunkyo  University 

Anne  Massey 

Indiana  University 

Eric  Monteiro 

Norwegian  University  of 
Science  and  Technology 

B.  Jeffrey  Parsons 

Memorial  University  of 
Newfoundland 

Nava  Pliskin 

Ben-Gurion  University  of 
the  Negev 

Jan  Pries-Heje 

Copenhagen  Business  School 

Arun  Rai 

Georgia  State  University 

Sudha  Ram 

University  of  Arizona 

Suzanne  Rivard 

Ecole  des  Hautes  Etudes 
Commerciales 

Rajiv  Sabherwal 

University  of  Missouri  -  St.  Louis 

Christopher  Sauer 

Oxford  University 

Peretz  Shoval 

Ben-Gurion  University 

Sandra  A.  Slaughter 
Carnegie  Mellon 

University 

Christina  Soh 

Nanyang  Technological  University 

Ananth  Srinivasan 

University  of  Auckland 

Kar  Yan  Tam 

Hong  Kong  University  of 
Science  and  Technology 

Bernard  C.Y.  Tan 

National  University  of 
Singapore 

Dov  Te’eni 

Bar-llan  University 

Yair  Wand 

University  of  British 
Columbia 

Richard  T.  Watson 
University  of  Georgia 

Gillian  Yeo 

Nanyang  Business  School 

Youngjin  Yoo 

Case  Western  Reserve  University 

ADMINISTRATIVE  PERSONN 

EL 

Eph  McLean 

Samantha  Spears 

Reagan  Ramsower 

AIS,  Executive  Director 

Subscriptions  Manager 

Publisher,  JAIS 

Georgia  State  University 

Georgia  State  University 

Baylor  University 

